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Introduction: A Comparative Network 
Analysis of Knowledge Use in Nordic 
Education Policies 


Kirsten Sivesind and Berit Karseth 


‘There is wide consensus in public policy today that globalization urges 
countries and their state authorities to bring scientific truth into policy- 
making processes. However, many scholars and policy analysts have also 
addressed ambiguity and complexity in regard to how scientific knowl- 
edge is used in policymaking processes (Cairney, 2016; Smith et al., 
2020; Steiner-Khamsi et al., 2020; Weingart & Guenther, 2016). Even 
though evidence is highly integral in terms of advanced, specialized 
knowledge, few scholars expect science and politics to serve the same 
roles and rationales. Moreover, the status and role of empirical evidence 
can be different between policy projects depending on the realms and 
knowledge these projects address, as well as their political contexts. 
Therefore, researchers have long expressed great interest in investigating 
how experts pursue their responsibilities and how they provide evidence 
in education policy (Heggen et al., 2010; Krejsler, 2013; Steiner- 
Khamsi, 2013). 


K. Sivesind (rx) ° B. Karseth 
Department of Education, University of Oslo, Oslo, Norway 
e-mail: kirsten.sivesind@iped.uio.no; berit.karseth@iped.uio.no 


© The Author(s) 2022 1 
B. Karseth et al. (eds.), Evidence and Expertise in Nordic Education Policy, 
https://doi.org/10.1007/978-3-030-91959-7_1 


2 K. Sivesind and B. Karseth 


In order to gain deeper insights into education policy and the way 
policymakers and experts deploy knowledge to legitimize school reforms, 
the authors in this edited volume examine citation patterns in corpuses of 
governmental papers written under the auspices of state authorities in the 
Nordic region of Europe. All empirical chapters are end results from the 
research project Policy Knowledge and Lesson Drawing in Nordic School 
Reform in an Era of International Comparisons (POLNET), funded by 
the Norwegian Research Council (NRC 283467). In this project, five 
Nordic research teams and a research team at Teacher College, Columbia 
University, New York, developed a shared database by systematizing ref- 
erences in policy documents. By collecting and analyzing bibliographic 
meta-data extracted from sources referenced in policy documents, the 
authors examine how these sources are tied together into networks of 
references in and across reform-making processes in five Nordic countries 
(Denmark, Iceland, Finland, Norway, and Sweden). Such sources include 
governmental documents and other types of work published by non- 
academic authors alongside research-based evidence produced at research 
institutions. In addition, we have interviewed key experts who have expe- 
rience from public inquiry bodies and ministry or agency officials with a 
particular responsibility for assisting leaders and members of the inquiry 
bodies. Some interviewees had also experience with referencing the inqui- 
ries in their work of authoring white papers for the government. 

In the POLNET study, our main interest is to examine how scientific 
knowledge and research-based evidence become affiliated with other 
knowledge sources in reference networks that legitimize policymaking 
processes. Our findings and interpretations result from a systematic anal- 
ysis of white and green papers. In the Nordic contexts, white papers are 
documents produced by governments to legitimize political recommen- 
dations that are reviewed and debated in parliamentary processes. These 
papers may have a profound impact on how discussions proceed in par- 
liamentary processes and how the bureaucracy and institutions within 
various sectors, such as education, eventually handle political problems. 
Due to the complexity of these problems as well as institutional demands, 
both politicians and governments are in a need of catalyzing changes and 
surveying people and practices to make sure that they adopt solutions 
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that benefit society. Therefore, these policymakers also ask for public 
inquiry reports, or what we in the volume call green papers. These green 
papers are most often written before a white paper is produced to evalu- 
ate issues and clarify the pros and cons of various options and solutions. 

In some countries, public inquiries are conducted by one senior officer 
within a ministry or by an external expert formally mandated by the min- 
istry. Ihis person reviews documents, collects data, and draws on various 
types of expertise to write a report about a particular topic. In the Nordic 
region of Europe, such inquiries are also made by advisory panels consist- 
ing of stakeholders, representatives from the sector, and/or experts who 
have the mandate to independently author a green paper to make recom- 
mendations for how to reform or renew policies in a particular area. 
These inquiries are organized to enlighten the work of policymakers and 
politicians and to inform the public, since they target political, social, 
cultural, and economic conditions of relevance for large parts of the pop- 
ulation in a country. Besides these inquires, ministries may ask experts to 
review a particular knowledge field and write a report without a 
government-issued mandate. In any case, debates based on these inqui- 
ries and expert reports can have various consequences. Formally, they 
may lead to political decisions within the parliament, which has the for- 
mal duty to represent the interests of the citizens in the country by mak- 
ing laws, legitimizing reforms, and overseeing the work of the government 
in terms of hearings, inquiries, and evaluations. Informally, discussions 
upon these reports can engage stakeholders and others in dialogues that 
make impact on education policy. 

A core aim of our analysis has been to discover differences between 
reference patterns in the countries white and green papers for the latest 
school reform related to compulsory education (grade 1—9/10), and to 
explore the role of knowledge usage in the policy processes that resulted 
in these papers. By drawing on Paul Cairney's (2016, p. 3) definition, 
“evidence’ is assertion backed by information" (p. 3), we regard the 
knowledge sources that are referenced as the actual evidence. Nutley et al. 
(2019) have provided an overview of evidence-promoting organizations 
and their ways of identifying, labeling, and ranking “good evidence,” on 
which the organizations have not reached any consensus yet. Thereby, we 
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consider the relative merits of various types of evidence as an open ques- 
tion for academic institutions and public branches, such as ministries and 
agencies, to have an ongoing debate about. 

A common view is that evidence-based policies draw on knowledge 
from outside of public policies, including from universities and research- 
based institutes where dedicated researchers produce scientific knowledge 
of relevance for policymakers. However, evidence and expertise are not 
only produced at universities and research institutes but also result from 
various activities and innovations across a variety of branches. These 
activities are arranged through networks of academic and non-academic 
partners that consist of public ministries, state agencies, non-governmental 
organizations, and even grassroots initiatives (Nelson & Campbell, 
2019). Various types of meetings, projects, and networks bring together 
partners in joint efforts to advance expertise that enlightens and legiti- 
mizes political actions and solutions. Researchers have offered good rea- 
sons to think of policy-relevant knowledge as developed across traditional 
division lines between science and politics (Godin & Shauz, 2016). In 
this context, policymakers and experts are expected to assess the evidence 
to determine “whether the source providing us with information is trust- 
worthy" (Eyal, 2019, p. 34) and relevant. Although evidence is expected 
to be scientific in a narrow sense, they may anyhow apply a broad concept 
of knowledge, including information, ideas, and arguments; well-tested 
beliefs; and lay, professional, and academic knowledge (Radaelli, 1995). 
Against this backdrop, relevant knowledge in public policy, authorized as 
a trustworthy source, can be broadly defined to include both academic 
work developed by scholars at universities or research institutes and 
knowledge that is regarded as less scientific because it is produced in non- 
academic contexts. Consequently, evidence can be based on or affiliated 
with various knowledge sources depending on where and how policy- 
makers and experts interact as well as how institutional reputation and 
legitimization serve as the modus operandi for the way policy is governed 
(Ball & Junemann, 2012). A key point in Steiner-Khamsi's (2013) work 
is that the production of knowledge, especially the design of comparative 
studies, optimizes evidence for executing at least three options of differ- 
ent types of knowledge use. That is, references can be used either (a) as 
evidence that informs policy planning within particular contexts, (b) as 
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normative guidelines for how to change educational processes concerning 
global problems, or (c) as projecting best practices that are evaluated 
against a set of international performance standards. There are good rea- 
sons for regarding the production of knowledge, especially the design of 
comparative research projects, as optimizing evidence for executing one 
or more of the three alternatives. The chapters that follow elaborate on 
these and related purposes by drawing on various theories as well as 
empirical data collected within the Nordic research project. 


Topics and Perspectives 


The POLNET study pays special attention to policy processes, in par- 
ticular to the nexus of national, regional, and international policy brokers 
and their knowledge provision and usage (Steiner-Khamsi, 2013). The 
combination of policy learning and borrowing (or reception) and the 
sociology of policy transfer serves as an analytical lens to study these top- 
ics. Drawing on globalization studies (Verger et al., 2018) and research 
on policy borrowing and lending (see, for instance, Steiner-Khamsi, 
2012; Waldow, 2012), the book's overall purpose is to enlighten a broad 
discussion on what counts as evidence for policymakers and which roles 
various types of expertise play in policymaking processes. Following 
chapters address one or several of the following research questions: 


* How do policymakers and experts in the Nordic countries (Denmark, 
Finland, Iceland, Norway, and Sweden) draw on domestic, regional, 
and international knowledge in their papers on school reform policy? 

* How do they legitimize national school reform policy by referencing 
various types of knowledge sources? 

* How do they authorize evidence and expertise in their attempt to pro- 
pose reform agendas, develop new or modified policy options, or issue 
new or revised school reforms in their respective countries? 


Our primary concern has been to explore the significance of evidence as 
a lever in policy formulation processes, notably by the prominence and 
role of single or individual references in terms of citations to specific 
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articles, book chapters, books, reports, and other documents. When 
source documents refer to clusters of references that share particular fea- 
tures, we can speak of spatial reference types. A central question that begs 
analysis is how policy processes provide reference patterns of both direct 
and spatial types within collections of governmental documents in areas 
of interest. 

It is our assumption that both single references and spatial reference 
patterns can project systems as well as outcomes. They can, for example, 
locate knowledge in different geographical entities (e.g., locally, nation- 
ally, globally) or refer to particular types of knowledge (e.g., books, jour- 
nal articles, governmental papers, reports) or to particular realms that are 
characterized by a set of thematic areas of interest. They can also project 
preferences for and prominence of certain types of policy analysis, such as 
systematic reviews, evaluations, sector analyses, and comparative studies 
in terms of international large-scale assessments that project best prac- 
tices in terms of evidence-based standards. By conducting comparative 
network analysis, there are many possibilities to identify both individual 
and spatial reference types that reveal insights into the way policymakers 
and experts co-construct knowledge that legitimize education policy. 

Another feature we have investigated relates to semantic patterns that 
can be extracted from content analysis of documents. The use of refer- 
ences and spatial patterns that evolve from policy processes results in 
thematic areas or policy realms that, in our case, relate to school reforms 
in the Nordic countries. These capture legislative reforms, curriculum 
and assessment policy, and policies addressing the development of liter- 
acy among children and adolescents. "Therefore, we also look into the 
policy texts by describing the content of reforms (see Chaps. 4, 5, 6, 7, 
and 8) and by unraveling core themes, decision strategies, and conceptual 
schemes that can be more or less outcome-oriented (see Chap. 7 in this 
volume). Eventually, we examine key narratives to discern how policy- 
makers and experts make sense of evidence provided by public inquiry 
bodies and other branches (see Chap. 12). 

The following three bodies of research have been important for design- 
ing sub-studies in the POLNET project that are included in this edited 
volume: (a) research on network governance; (b) studies of “travelling 
reform’ (i.e., diffusion vs. reception studies); and (c) research on 
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evidence-based policy. In the next section, we will briefly present each of 
these bodies. 


Research on Network Governance 


First, this book draws on research about the role network governance 
plays in education policies and how changing conditions for state regula- 
tion can help with interpreting variations in reference patterns. Several 
researchers have pinpointed the recent transformation of network gover- 
nance that extends new public management policies introduced in most 
European countries. Helgoy et al. (2007) have described the contempo- 
rary changes in education policy as a re-regulation of the society by ways 
networks of actors make use of new governance tools. In the era of com- 
parative education analysis, Ball and Junemann (2012) have contributed 
to a renewed understanding of how policy networks are changing educa- 
tion policy by including philanthropy and business partners in public 
policymaking. Combining social network analysis and ethnographic 
research, the authors unraveled how new communities of social actors, 
such as think tanks and interest organizations, influence public policy by 
actively engaging in conversations beyond institutional procedures. By 
referencing political scientists (Bevir & Rhodes, 2003), Ball and 
Junemann (2012) argued that there is a shift from government to gover- 
nance, or from a unitary state where government is "carried out through 
hierarchies or specifically within administrations and by bureaucratic 
methods" to governance, which is "accomplished through the 'informal 
authority’ of diverse and flexible networks" (p. 168). Moreover, these net- 
works move beyond the methods of new public management by being 
configured in terms of tightly connected policy communities or more 
loosely connected issue networks. Such networks arise from the exchange 
of resources, such as money, information, or expertise that enable policy- 
makers to accomplish their goals. 

Our book contributes new insights about the development, use, and 
reception of expertise in public policy. Building upon Ball and Junemann’s 
(2012) theorizing about network governance, we extend the idea of 
loosely coupled issue networks. By looking for spatial reference patterns 
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in policy documents, we explore how network governance underpins 
agenda setting and policy formulation processes in reform areas such as 
curriculum and assessment policies. Certainly, both of these areas are 
emphasized in Nordic school policy today and have roots in reform tra- 
jectories, developed by professional communities from the early twenti- 
eth century and onward. Throughout the 1900s, both intellectuals and 
reformists developed knowledge and expertise about assessment that 
resulted in progressive reform movements in the Nordic countries (Ydesen 
et al., 2013). More recently, international comparisons have been con- 
ducted in all Nordic countries (Sivesind, 2019) as a key monitoring tool 
by the national governments to make policy decisions and assess the qual- 
ity of teachers, schools, districts, and the education system itself (Proitz 
et al., 2017; Camphuijsen et al., 2020; Skedsmo et al., 2020). These com- 
parisons enable international organizations to serve as policy actors and 
knowledge providers, which have implications for how national assess- 
ment systems are designed (Tveit & Lundahl, 2018) and how curriculum 
policy is formulated and enacted under the auspices of national govern- 
ments (Molstad & Karseth, 2016; Nordin & Sundberg, 2016; Sivesind 
& Wahlström, 2016). 

However, countries differ in the expansion of non-state actors in their 
educational policy processes. Ihompson (2003) explained this difference 
by noting that networks result from interactions between various compo- 
nents, such as ideas or concepts, people, institutions, social practices, or 
bodies of knowledge, and thereby constitute a form of "assemblage." 
Moreover, networks are also characterized by a feedback loop mechanism 
that transforms policies by their own logics that more often than not 
result in configurations that connect hierarchies with the market (p. 363). 
While researchers in both political and educational sciences first assumed 
that network governance was a reform strategy that changed the way gov- 
ernments could govern public services, several scholars have highlighted 
the fact that this strategy does not necessarily transform democracies in 
all that matters. As a result, network governance does not necessarily 
change the hierarchies of bureaucracies into a totally new shape. Based 
upon this discovery, researchers have begun to consider networks as being 
characterized by hierarchies in some form. To what degree bureaucracies 
are shaped by network governance is thereby an empirical question. 
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This notion led to new insights about the role of the state in public 
administration and the idea of hybrid forms of governance, which is a 
highly contested concept as well. Renowned scholars in the field of edu- 
cation and political science have questioned whether public policy has 
transformed into a post-bureaucratic mode of governance (Maroy, 2012). 
This notion motivated Philippopoulos-Mihalopoulos (2012, p. 12) to 
write, ^Ihere is no such thing as a hybrid (yet)" (p. 12), meaning that 
there can be hybrid governance modes present in public policies, but not 
necessarily everywhere. The same author argued that the appellation 
hybrid more often than not refers to pre-existing connections with “a 
certain otherness" (p. 12). Thus, formal bureaucracies that are hierarchi- 
cally organized and governed by formal and substantial rationales in the 
Weberian way can still survive within a network-based society; however, 
they are observed differently within the context of post-bureaucratic gov- 
ernance (Hall & Sivesind, 2015). For example, Theisens et al. (2016, 
p. 467) argued from the perspective of New Public Governance that the 
state still plays a dominant role in the political context where public sec- 
tor organizations operate as knowledge providers. This is certainly true 
for the Nordic countries, which in many ways moved beyond the era of 
new public management during the 1980s (Christensen & Lzgreid, 2007). 

Following the same argument, Greve et al. (2016, p. 192) argued that 
reform policy today seems to emphasize state-centered solutions while 
acknowledging the mix of governance mechanisms and institutional 
complexity that characterizes the public sector. They concluded that state 
bureaucracies are still operative institutions that govern reform processes 
in the Nordic countries, but they are "less driven by politicians, more 
planned, and less contested by the unions, and there is more public 
involvement" (p. 201). Consequently, public policy is still observed as 
shaped by hierarchical structures in terms of government; however, it is 
observed differently than before, due to evolving networks that are re- 
ordering the environment of bureaucracies. This situation has generated 
renewed interest in collaborative governance. In this volume, authors 
argue that there are many more reform actors on the international stage 
who pursue their own strategies and reform activities besides national 
governments; furthermore, this situation does actually lead to deregula- 
tion in particular policy realms “as a process of removing or reducing state 
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regulations" (Dovemark et al., 2018, p. 123). Against this backdrop, 
there are good reasons for looking at the orchestration of global gover- 
nance and at international organizations as intermediary bodies that 
influence national governments (Abbott et al., 2015). 


Studies of Traveling Reform: Diffusion Versus 
Reception Studies 


Second, in part because of new relations and connections between local 
and global venues for policymaking, authors in this volume draw on aca- 
demic research about the increased reliance on externalization in educa- 
tion policy. This theory is based on a sociological theory about how 
systems, such as education, politics, and science, interplay within a world 
characterized by increased complexity. The theory pinpoints the impor- 
tance of researching how education policy is legitimized (Steiner & 
Waldow, 2012), and therefore we are examining how policymakers refer- 
ence international and regional sources and/or cross-sectorial works to 
justify reforms in their Nordic countries. This dimension, which focuses 
on the legitimization of reforms, helps us to explore how policy processes 
change character through policy transfer (i.e., traveling reforms) and, in 
particular, if and how policymakers and experts borrow solutions from 
abroad or draw lessons about how to reform education in specific con- 
texts (Steiner-Khamsi, 2012). 

A group of institutional researchers studying traveling policy looks at 
the diffusion and dissemination of international standards or global edu- 
cation regulations as the key mechanisms for researching the transfer of 
policy knowledge. Their views enable them to understand why certain 
policies are expanded and promoted while others are not. Obviously, 
their views allow us to understand the active role of international organi- 
zations in lending or disseminating specific policies and programs, which 
are referred to as "best practices" or "international standards" (Bromley & 
Meyer, 2015; Krücken & Drori, 2009). However, this "from above" view, 
which is often associated with the theory of world culture, provides only 
one of several perspectives for understanding the dissemination of global 
education policy. Another angle clarifies how, why, and when national or 
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local policy actors selectively learn from global education policies based 
on local conditions (Baek et al., 2018). When borrowing global educa- 
tion policies, such as competency-based curriculum reforms, account- 
ability reforms, or public-private partnership policies at the local or 
national level, this “bottom-up perspective" focuses on the process of 
both reception and translation (Steiner-Khamsi, 2015). 

In the past few years, more studies have emerged that have advanced 
the “look from below" view in important ways. The Scandinavian insti- 
tutionalist approach belongs to this tradition, which differs from other 
institutional research traditions by being concerned with non-strategic 
approaches and involving researchers who proclaim that ideas and prac- 
tices undergo profound changes when deployed in new organizational 
settings (Czarniawska & Sevón, 1996; Rovik, 1996; Sahlin-Andersson, 
1996). In this theory, stories, images, narratives, and master ideas serve as 
lenses to think of policy transfer that is shaped globally without necessar- 
ily attracting everybodys attention or direct involvement across the globe. 

‘There is also a long tradition in comparative education of studying the 
reasons for "transnational policy attraction" within local and regional 
contexts, that is, to study why the government borrowed, imitated, or 
transferred others’ policies (Phillips, 2004). Similarly, researchers have 
studied the role of public-private partnerships and privatization in such 
contexts to examine the question: Why does global education policy res- 
onate in a specific context; in other words, why do policy participants 
“buy” the policy? Verger (2014) has offered an explanatory framework to 
understand how reform plans are “sold” to the governments of low- 
income countries (Verger et al., 2016). In this case, an economic theory 
helps to explain why projects and plans are bought by non-affluent coun- 
tries. Recent studies have also asserted that global education policies reso- 
nate for different reasons in different contexts, however, not necessarily 
due to the logic of quasi-markets (Maroy & Pons, 2019). The idea of 
evaluative state and international organizations as standard-setters makes 
sense for understanding why international large-scale assessments 
(ILSAs), such as the Programme for International Student Assessment 
(PISA) and the Trends in International Mathematics and Science Study 
(TIMSS), resonate differently in different countries (Addey et al., 2017; 
Martens et al., 2010; Pizmony-Levy, 2017; Ydesen, 2019). Nonetheless, 
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the reception and use of ILSAs are most often dependent on how national 
policies project their own goals and conceptions of best practices (Waldow 
& Steiner-Khamsi, 2019). 

Verger (2014) emphasized that policy ideas need to be placed into a 
context for understanding how global policies get transformed once they 
have penetrated a local arena. Political institutions, administrative and 
regulatory viability, government ideology, domestic political contention, 
and periods of crisis create a context for strategic selectivity regarding the 
reception of policy ideas. Maroy and Pons (2019) demonstrated the 
importance of context in understanding how transnational models of 
accountability are recontextualized and translated into national policies 
in various ways. Furthermore, Karseth and Solbrekke (2010) analyzed 
the Bologna policy and the translation of the European Qualifications 
Framework in terms of how national qualification frameworks have been 
developed in different countries. All the powerful institutions within 
these countries examined, emphasized the importance of learning out- 
comes as a reference point; however, they interpreted the Bologna policy 
differently, as they created various windows of opportunities for reform 
and change within their respective contexts. 

In European countries and elsewhere, policy knowledge has initiated 
changes in various directions. On the national level, civil society organi- 
zations, companies, and local associations have expanded their influence 
in public education. On the transnational level, international organiza- 
tions have evaluated, compared, and ranked education systems, produc- 
ing various diffusion mechanisms that have made non-state actors into 
standard-setters (Peters et al., 2009) and by advocating what Steiner- 
Khamsi (2013) characterizes as “best practices.” Moreover, scientific 
communication itself, mediated by international journals and their media 
strategies, legitimizes reforms in the education sector (Molstad et al., 
2017). Although non-state-actor organizations have no formal mandate 
to govern education reform, they still execute informal power through 
new “soft” but effective knowledge products within policy networks that 
are often characterized by a particular language, where measurements in 
terms of calculations and numbers serve as the core instrument (Grek, 
2008). As a result of new technology for governing education, policy 
processes have come to be configured by the production and use of 
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knowledge (Abbott et al., 2015; Bregger, 2018; Fenwick et al., 2014; 
Littoz-Monnet, 2017; Peters et al., 2009). 

One approach to grasping "travelling reforms" and the use of refer- 
ences to foreign educational systems is, as already mentioned, the exter- 
nalization thesis. According to Schriewer (1988), references to external 
systems (i.e., externalization) are mobilized to add meaning, weight, or 
legitimacy to domestic reforms. Renowned scholars have used the sys- 
tematic theoretical concept of externalization to explain the selective bor- 
rowing of global education policies, the popularization of international 
best practices, and the transfer of reforms from one country to another 
(Schriewer & Martinez, 2004; Steiner-Khamsi, 2003, 2009, 2012, 2021; 
Waldow, 2012). Likewise, Nordic researchers have contributed to research 
on policy transfer in the field of education policy and applied semantic 
analysis that draws on sociological system theory (Luhmann, 1990, 
1997). They show how national public inquiry reports create their own 
projections of the future school as a contemporary way to manage co- 
existing expectations (Hansen et al., 2021) and how national curricula 
reconfigure the scientific logics of comparative assessment designs that 
characterize transnational policies (Sivesind et al., 2016). 


Research on Evidence-Based Policy 


Third, we discuss how evidence-based policy is anchored in particular 
forms of knowledge that legitimize school reforms, education policies, 
and practices. As a field in political science, the politics of knowledge 
focuses on the legitimacy of knowledge forms and institutional logics 
that are shaping, for example, the evaluation-knowledge nexus 
(Segerholm et al., 2019). Simultaneously, theories of the policy process 
more generally focus on various features of evidence-based policies and 
practices. Together, these fields reflect a rather weak interest in knowledge 
use (Daviter, 2015). Moreover, several studies have examined what can be 
used as evidence, usually as the foundation of knowledge-based policies 
and regulations. Kvernbekk (2011) recommended a deep look into the 
question of the nature of evidence and how it is viewed and conceptual- 
ized in academic scholarships; however, both previous and recent 
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literature has acknowledged the use of multiple types of evidence in the 
policy process beyond what is considered validated by scientific proce- 
dures (Boaz et al., 2019; Weiss, 1979). 

Despite doubts about how policy synchronizes its projected solutions 
and scientific evidence accordingly, a considerable number of knowledge 
providers have written books, articles, reports, and policy briefs to aid the 
development of evidence-based policy. Some of these knowledge provid- 
ers have sought to single out “paths forward, toward bringing about 
more/better evidence use in education" (Malin et al., 2020, p. 11); as a 
result, evidence-based policy is regarded as a campaign-like adventure, or 
at least a master idea or a trend (Rovik, 2016). By pointing to various 
challenges that policy experts often experience, researchers have also 
raised doubts about the scientific validity of the knowledge sources pro- 
duced and used throughout policy processes (Holst & Molander, 2018). 
Although academic scholars are more frequently involved in public 
inquiries (Christensen & Hesstvedt, 2019), the institutional logics and 
contextual conditions that regulate policymaking processes are not neces- 
sarily resulting in scientific practices as they normally conducted in uni- 
versities or at research institutes. Policymaking processes follow their own 
institutional logics and can be seen as more pragmatic in terms of how 
expertise apply scientific procedures and standards. Moreover, as Cairney 
(2016) emphasized, the environments of decision processes are character- 
ized by complex systems where connections between actors and agencies 
are many and varied. Due to a lack of centralized control that follows 
from the way society develops, there is no guarantee that evidence will 
actually be used even when it is provided by experts (Cairney, 2016). 

Thus, the definition of evidence is broad and highly contested; conse- 
quently, researchers and policy analysts frequently discuss what consti- 
tutes "good" evidence. Moreover, evidence cannot be researched without 
insights into the contexts where it is used. In Chap. 2, Steiner-Khamsi 
presents a theoretical overview of the relevant definitions and literature 
that enlighten the various features of evidence-based policy in the educa- 
tion sector. In particular, she outlines how the definition of evidence 
(depending on the situation and the stage of the policy process in our 
case) relates to the kind of reform policies we have explored in the 
POLNET project. In the next sections of this chapter, we will provide a 
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brief overview of the data and methods we draw on in the following 
national and comparative chapters and give a brief overview of the Chaps. 
2, 5, 4, 5, 6, 7, 8, 9, 10, 11, 12, and 13. 


Significance of the Data and Methods 


To examine evidence-based policy, the POLNET study applies biblio- 
metric and network analysis to trace the reception of knowledge sources 
and their formation into reference networks. A core purpose has been to 
explore differences in reference patterns extracted for a collection of 
source documents that legitimized the most recent school reforms for 
compulsory education in five Nordic countries. In this method section, 
we will provide an overview of the database we have searched, including 
the search strategy and the date we collected the data, the selection pro- 
cess, and the number of records retrieved in order to make our visualiza- 
tions reproducible. Against this backdrop, each empirical chapter covers 
the analytical strategies we have pursued in more depth. 

We consider both bibliometric and network analysis to be useful and 
versatile tools that provide insights into what Eyal (2019, p. 33) labeled 
"distributed cognitions of expertise" (p. 33); in our case, this refers to 
how expertise outside individuals is visualized through bibliometric refer- 
ence patterns. In order to interpret and make sense of the patterns visual- 
ized in terms of quantitative measures and graphs, we have also interviewed 
policymakers and experts as well as synthesized and examined the con- 
tent of core documents. Moreover, in all chapters, we draw on contextual 
information to analyze the reception and translation. of knowledge 
sources within our source documents. We combine quantitative and 
qualitative approaches to provide a rich and comprehensive understand- 
ing of the cases we look into and to explore differences in citation patterns. 

The origins of this book lie in the joint research project POLNET, 
which enabled six research teams to systematize references in policy docu- 
ments to create a shared bibliometric network database. Based on a joint 
protocol developed by the research team at Teacher College, Columbia 
University, New York, each team selected a comparable set of policy doc- 
uments and associated sources that were produced in conjunction with 
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the last national school reforms in the five Nordic countries. These 
reforms addressed broad themes, such as renewing the national curricula, 
redesigning assessment systems, and amending the legislative system. 

An overall goal was to measure the same phenomenon, specifically offi- 
cial/state policy knowledge used to set reform agendas, develop new or 
modified policy options, or issue reforms for basic education (Years 
1—9/10), and to focus on the same variables, that is to enter relevant 
information on the references (e.g., year of publication, location of pub- 
lication). We used sampling plans and a shared template for how to enter 
data to guarantee consistency. In the Nordic group meetings, we agreed 
that the comparative study would use a sampling strategy that approxi- 
mates the one used for the Norwegian study (see Baek et al., 2018). In 
particular, the selected source documents should reflect official/state pol- 
icy knowledge that the government had used in preparation for a particu- 
lar school reform; in other words, these were white papers, green papers, 
or functional equivalents of these Norwegian documents. Once all the 
data from the five research teams were entered, the Teacher College, 
Columbia University-based research team cross-checked the entered data 
in collaboration with the researchers. In addition, all the researchers par- 
ticipated in joint discussions during Nordic research meetings about 
challenges and solutions related to data entry, cleaning, and coding. In 
this way, we ensured that the data were comparable. For an overview of 
the source documents, see Chap. 3 (Table 3.2). 

The joint protocol outlined the sampling method (i.e., the selection of 
official/state policy knowledge documents) and the data entry protocol 
(i.e., the identification of variables used in the databases). We regard the 
documents as reflecting both the artifacts and discourses produced by 
policy actors, whereas the references in those documents reflect the evi- 
dence base for these artifacts and discourses. Additionally, the references 
in policy documents show the linkages between policy documents; for 
example, if two policy documents draw on similar evidence, they share 
some kind of affinity. By applying network analysis, we analyzed how 
policymakers and experts recognized the same citations and references 
across thematic areas and contexts. This view helps us to interpret the 
various knowledge networks they build, being based on both proximity 
and distance, respectively. Furthermore, since our cases are located in the 
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same geographical region of Europe, we explore to what extent school 
reforms in these countries draw on shared knowledge sources and, more 
generally, whether authorization of knowledge sources differs across con- 
texts. Table 1.1 summarizes the reforms examined in the POLNET study, 
the number of source documents, and their references. 

After extracting the bibliometric network data on the relationship 
between source documents and references from each source document, 
we entered a series of attributes for all documents in the database to allow 
for interpretation. These attributes are as follows: (a) year of publication; 


Table 1.1 Reform titles and data 


Total 
number of 
Number Number source Number of 
Reform of WPs of GPs documents references 


Denmark The Danish Public 1 4 5 251 
School Reform of 
2013 
Finland National Core 1 9 10 677 
Curriculum for 
Basic Education, 
Reform of 2014 
Iceland The Renewal ofthe 2 2 4 203 
Fundamental 
School Reform, 
Reform of 
2013/2017 
Norway The Renewal ofthe 2 8 10 2645 
Knowledge 
Promotion Reform, 
Reform of 
2016/2020 
Sweden A Gathering for 1 8 9 1615 
School—National 
Strategy for 
Knowledge and 
Equivalence, 
Reform of 
2015/2018 


Note: The number of references refers to the sum of every reference in all source 


documents. It counts the references that are cited by multiple sources separately. 
WP: White Papers. GP: Green Papers 
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(b) publisher or institutional affiliation of the author/authoring organiza- 
tion; (c) location of publication (i.e., domestic, regional, and interna- 
tional); and (d) type of publication (i.e., report, book, journal article, 
government-published document, and other). We used the attribute 
information to comprehend the networks and reference utilization pat- 
terns within and across the countries. 

In this project, we focused on one of the major network measures (i.e., 
in-degree centrality, which equals the total of incoming citations for a 
given document) to assess the importance of a reference in a bibliometric 
network. We have also used other measures, such as co-citations, to iden- 
tify policy networks (i.e., authors within the ministries or public inquiry 
bodies that cite each other or that cite the same texts) and peripheral 
texts/discourses that bridge two or more networks. The software pro- 
grams UCINET 6.289 and NetDraw 2.097 were used for network analy- 
sis and visualization, respectively (Borgatti et al., 2002). 

In addition to bibliometric network analysis, several chapters in this 
book include additional data. Text analysis is essential for interpreting the 
quality of the relations between referenced knowledge sources at levels 
beyond what can be recognized by individuals within the policy context 
and what can be perceived and potentially developed. The content analy- 
sis of the white and green papers has enabled the research teams to trace 
and compare reception and translation processes—that is, to explore 
which concepts and statements of a national, regional, or international 
text resonate and how they are reframed or translated in a national setting. 

The semi-structured interviews we have added to complement the bib- 
liometric network analysis helped us make sense of the reference patterns. 
During in-person and Zoom interviews, a group of 20 policymakers and 
experts involved in reform-making processes in 4 of the given countries 
shared their experiences and knowledge about reference practices in and 
across policy areas and realms. 'Ihese interviews covered the same themes 
for the purpose of synthesizing and analyzing findings based on the inter- 
view transcriptions. Examining their responses has allowed us to explore 
why knowledge is used differently between the Nordic countries. We col- 
lected data according to the General Data Protection Regulation, set by 
the European Union, and followed ethical guidelines. 
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Throughout our POLNET study, we have applied bibliometric analy- 
sis, which have been primarily utilized in the field of information science 
(Gingras, 2016). Policy analysts have used these instruments to evaluate 
education research and its impact, which is particularly known from 
research policy in the United Kingdom (Smith et al., 2020). However, 
few researchers have applied bibliometric analysis to conduct compara- 
tive and international research studies of policy knowledge in education 
reform projects and critically examined what these methods can be used 
for. This book contributes to the scholarship by applying bibliometric 
analysis to examine citation patterns in policy documents and theorize 
the policy process from a comparative perspective. 


Chapter Overview 


In Chap. 2, Gita Steiner-Khamsi provides a theoretical framework for 
how to examine knowledge use to understand how evidence-based policy 
evolves and changes within contemporary practices. She argues that the 
unit of analysis for bibliometric analyses is the reference, which may be 
listed conveniently at the end of a document in a separate bibliographical 
section, in a footnote or endnote, or, more inconveniently, only vaguely 
alluded to in a text. In either case, the reference communicates or conveys 
something to the reader. The question then becomes what exactly it com- 
municates; in the context of the POLNET study, this raises the following 
question: How have we interpreted the reference in the larger corpus of 
the policy documents that we collected in Denmark, Iceland, Finland, 
Norway, and Sweden in order to understand the policy process? 

For our bibliometric analysis of evidence-based policymaking in the 
Nordic region, Steiner-Khamsi found Paul Cairney's (2016, p. 3) defini- 
tion useful: “evidence” is assertion backed by information” (p. 3). In con- 
cert with Cairney’s definition, Steiner-Khamsi argues that references are a 
construct or an aggregate of several pieces of information (e.g., author- 
ship, year of publication, topic, or theme) that help position the author 
in a larger semantic space. According to Steiner-Khamsi, all these consti- 
tutive elements are essential in bibliometric analysis, as they are utilized 
as epistemological cues for understanding not only whose texts or whose 
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knowledge the authors have selected to substantiate their points but also 
whose knowledge they cite as sources of expertise to reduce uncertainty 
or generate legitimacy about the validity of their own claims or assertions. 
Steiner-Khamsi recommends drawing on the systems-theoretical delib- 
erations presented in Gil Eyal's (2019) book The Crisis of Expertise, and 
what Baek (2020) labeled the "expertise-seeking arrangements" in the 
five Nordic countries. Finally, Steiner-Khamsi invites further research 
that helps to refine the framework and the method of inquiry, including 
investigating the hierarchization of evidence as reflected in the choice of 
references. 

In Chap. 3, Oren Pizmony-Levy and Chanwoong Baek describe the 
methodological approach behind the project. The authors discuss how 
the research design is theoretically and methodologically inspired by 
existing literature on social network analysis and sociology of knowledge. 
They demonstrate how bibliometric network analysis that is grounded in 
these two lines of research allows us to examine the architecture of policy 
knowledge in the five Nordic countries. The authors also detail the pro- 
cedures for collecting, analyzing, and presenting data for this project and 
provide the rationale for drawing conclusions based on bibliometric 
analysis. 

Chapters 4, 5, 6, 7, and 8 represent national case studies from the 
Nordic countries where bibliometric data are used to shed light on the 
policy processes behind recent school reforms. They all examine the use 
of references in key documents, such as white papers and green papers, 
and address how school reforms are being legitimated in certain ways 
through alignments of evidence produced on domestic, regional, and 
international levels. To contextualize the reform, several of the studies 
combine data. 

In Chap. 4, Trine Juul Reder and Christian Ydesen analyze the evi- 
dence base and policy context of the Danish public school reform of 
2013. In addition to the bibliometric data, the study draws on a contex- 
tual reading of policy documents and interviews with informants who 
played a key role in the policymaking process. The findings indicate that 
the reform was based on a quite limited number of written sources and 
left a great role to be played by more informal knowledge. Overall, the 
chapter categorizes the evidence base into academic knowledge, strategic 


1 Introduction: A Comparative Network Analysis of Knowledge... 21 


knowledge, and knowledge produced by international organizations, 
such as the OECD. Furthermore, the authors also identify stakeholder 
knowledge and practice-based evidence as additional categories. 

In Chap. 5, Saija Volmari, Jaakko Kauko, Juho Anturaniemi, and Íris 
Santos examine what kind of evidence Finland draws on in the policy- 
making process of the 2014 national core curriculum. Based on biblio- 
metric and content analysis, the authors show that the most influential 
international policy documents in terms of policy design came from the 
OECD; however, the references used were mostly domestic. Hence, while 
the global level has the power to produce evidence, the national or local 
level experts, who build their influence through networks, have the power 
to select the evidence and adjust it to meet national needs. The authors 
conclude that the type of evidence, merely consisting of empirical evalu- 
ation data, appeared to be more important than where the evidence origi- 
nated from (local or global level). This means that a global policy space 
intervened with the national. 

In Chap. 6, Magnúsdóttir and Jónasson explore the formation of pol- 
icy documents that were issued after the school reform the national school 
authorities in Iceland organized during 2008-2013. The main emphasis 
is on the first document issued as a white paper (2014) by Icelandic state 
educational authorities, while the two other policy documents were 
framed by international organizations. In addition to bibliometric analy- 
sis, a content and a interview study with with five experts, confirm a min- 
imal use of academic references and unsystematic development of green 
papers. Apart from a shared focus on enhancing quality in education, 
there is a low interconnection between the documents in terms of con- 
tent, bibliography, and semantics. Only one of these three source docu- 
ments was written in Icelandic, and therefore the Icelandic case is 
exemplary in terms of externalization with robust references from the 
OECD. The reforms short timeframe shaped by the aim of the Minister 
of Education was to react promptly to declining results in PISA. The dif- 
ferent formulations manifested by these documents reveal an eclectic 
approach to knowledge usage shaped by ministerial governance. 

In Chap. 7, Bernadette Hórmann and Kirsten Sivesind present a bib- 
liometric and semantic analysis of the use of knowledge sources refer- 
enced in two white and eight green papers about the recent school reform 
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for primary and secondary education in Norway (2016/2020). The 
authors identify the most often co-cited texts in the bibliographies and 
the most frequent in-text references in the two white papers. Based on 
Luhmann’s (2000) distinction between conditional and purposive pro- 
grams, the authors examine how policymakers use the prominent refer- 
ences, classified into four types, as evidence to propose a set of policy 
options. The four groups are: (a) formal documents that are not primarily 
a result of research but serve as one type of evidence, (b) meta-analysis, 
(c) configurational reviews, and (d) empirical research studies. The chap- 
ter concludes that meta-analysis and configurational reviews support pur- 
posive arguments along with means-end reasoning. In addition, the 
authors find that formal documents, with some exceptions, serve a regu- 
latory role in terms of their conditional orientation, while empirical 
research reports, originally developed for evaluative purposes, help poli- 
cymakers to align systems of reasoning depending on the issues they 
address. 

In Chap. 8, Andreas Nordin and Ninni Wahlstróm examine the selec- 
tion and use of evidence in the most recent school reform in Sweden, the 
Knowledge Achievement Reform (2015/2018). The analyses of citation 
frequency show that, although the OECD had an important role in ini- 
tiating the Swedish reform, the highest percentage of references was 
domestic and mainly governmental references. This shows the possibility 
for national politics to uphold a high level of self-referentiality even when, 
to a large extent, international organizations such as the OECD suggest 
the national political agenda. Another distinct feature the authors empha- 
size is the low number of academic references. 

Unlike the individual country studies in the previous chapters, Chaps. 
9, 10, 11, and 12 address four comparative topics and bring together 
researchers from the different national teams. Chapter 9 compares the 
knowledge governments use to inform their policy decisions. Chanwoong 
Baek, Dijana Tiplic, and Íris Santos show that all five Nordic countries 
actively utilized knowledge to support and legitimate their policy propos- 
als. However, they did so in different ways and in different settings. Some 
Nordic countries sought evidence for policy proposals mainly through 
the policy advisory system within the bureaucracy, while others out- 
sourced the production of policy advice. Furthermore, the analysis 
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showed that reference utilization depends on the extent to which the 
policy system is self-referential or receptive to externalization. The authors 
also highlight the differences regarding whether the reform can be 
described as an incremental or a fundamental reform. A fundamental or 
controversial reform often utilizes more international references than an 
incremental or non-controversial reform. 

In Chap. 10, Gita Steiner-Khamsi, Chanwoong Baek, Berit Karseth, 
and Andreas Nordin compare reference patterns between Norwegian and 
Swedish green papers (cross-national comparison), as well as between 
green papers and white papers (political translation). The authors con- 
clude that the advisory commissions have been repurposed in ways that 
place greater emphasis on expertise rather than accountability and repre- 
sentation. Second, the multi-level analysis shows that the commissions 
merely represent one stage in a long sequence of evidence-based policy- 
making. Finally, by following a transnational perspective, the chapter 
shows how advisory commissions today are used as bridges between the 
global and the national. 

Chapter 11 draws attention to the OECD’s role in the policy process 
in Nordic countries. Christian Ydesen, Jaakko Kauko, and Berglind Rós 
Magnúsdóttir investigate the extent to which the OECD and national 
institutions function as data-driven knowledge brokers in the shaping of 
education in the Nordic region. The chapter offers an in-depth analysis of 
Denmark, Finland, and Iceland to understand the field of knowledge 
brokers in general and the role of the OECD in particular. The chapter 
shows how policy flows via its illumination of the configuration and 
workings of the OECD-centered epistemic community, forming the 
modes of knowledge and governance woven into the fabric of Nordic 
education. 

Chapter 12 considers how the regional level works as a reference in 
national education policy planning. By drawing on bibliometric analyses, 
content analysis, and interviews, Saija Volmari, Kirsten Sivesind, and Jón 
Torfi Jónasson examine the actual role and influence of Nordic knowl- 
edge and cooperation in recent school reforms. The bibliometric analysis 
uncovers an extremely low number of regional references in the actual 
policy documents. However, drawing on the interview data, the authors 
conclude that both global policy spaces and policy places in local settings 
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provide regional evidence of relevance for policymaking processes. Nordic 
interrelations and interactions are experienced, meaningful, and impor- 
tant; however, not made visible using specific bibliometric sources but 
rather more indirectly by translating ideas about the “Nordic” or other 
Nordic countries. 

In Chaps. 13 and 14, two renowned scholars, Kerstin Martens and 
Antoni Verger, contribute by commenting on the book's overall theme 
and proposing what they regard as significant problems that research on 
evidence-based policy should address in the future. Finally, in the con- 
cluding chapter, Berit Karseth and Kirsten Sivesind synthesize and inter- 
pret findings from the Nordic POLNET study. 
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What Is in a Reference? Theoretically 
Understanding the Uses of Evidence 
in Education Policy 


Gita Steiner-Khamsi 


Arguably, the main unit of analysis for all bibliometric analyses is the 
reference. It may be listed conveniently at the end of a document in a 
separate bibliographical section, in a footnote, in an endnote or, more 
inconveniently, only vaguely alluded to in the main text. In either case, 
the reference communicates or conveys “something” to the reader. One 
question arises: What exactly does it communicate, what is embedded in 
a reference, or, to place the question into the context of our study, how 
have we interpreted the reference in the larger corpus of policy docu- 
ments that we collected in Denmark, Iceland, Finland, Norway, and 
Sweden in an attempt to understand the policy process? 

References have several constitutive elements: an indication of author- 
ship, year of publication, topic or theme, location of publisher, and type 
of publisher. The authorship may be further differentiated by gender, 
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nationality, institutional affiliation, and, if several coauthors are involved, 
the network structure within the group of authors. All these constitutive 
elements are essential in a bibliometric analysis because they are utilized 
as epistemological cues for understanding not only whose texts or whose 
knowledge the authors have selected to substantiate their points, but also 
whose knowledge they cite as sources of expertise to reduce uncertainty 
or generate legitimacy about the validity of their own claims or assertions. 

According to political scientist Paul Cairney (2015), “‘[elvidence’ is 
assertion backed by information." In our bibliometric network analysis, 
we treat references as a construct or aggregate of several pieces of informa- 
tion (authorship, year of publication, topic or theme, etc.), helping posi- 
tion the author in a larger semantic space. Referencing sources in a policy 
document—whether it is a Green Paper (in the Nordic context: prepared 
by government-appointed expert panels) or a White Paper (issued by the 
government)—is fundamentally different from the references in, for 
example, an encyclopedia, where the expectation is that the researcher at 
least pretends to identify the universe of relevant research literature on 
the topic with which they can situate their own framework. Policy docu- 
ments are much more evaluative and controversial in nature; therefore, 
they tend to openly take a stance for or against existing assertions. 
Unsurprisingly, a bibliometric analysis of policy documents will often 
surface clusters of like-minded authors/documents and set them apart 
from those holding viewpoints that are seen as different or distant. 


References from the Perspective 
of Sociological Systems Theory 


In the Nordic POLNET (Policy Knowledge and Lesson Drawing in an 
Era of International Comparison) study, we interpret a reference func- 
tionally, that is, in its larger context of evidence-based policy planning. 
We ask the following: What does a reference stand for—or rather do—in 
a policy document? From the perspective of sociological systems theory, 
references are meant to reduce uncertainty for the reader. They do this by 
making transparent the positionality of the author in the larger 
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discursive policy space and by documenting the credibility of the sources 
used for the assertions the author is making. As Jenny Ozga (2019) has 
astutely pointed out, the discursive space in policy documents is by 
default a political space in which authors position themselves in terms of 
political orientation and alliances. 

To reiterate, references help validate or provide legitimacy to the evi- 
dence that the author (e.g., the government-appointed expert commis- 
sions or the government) has presented in the document. Thus, if 
"evidence is assertion backed by information” (Cairney, 2015), then a 
reference is validation of evidence. Said differently, references are used to 
provide authoritative status to the evidence presented in policy docu- 
ments. Naturally, a host of questions surface with this particular concep- 
tualization of references: Which texts are influential, that is, referenced 
frequently or referenced by two or more different knowledge networks? 
Are international references, that is, texts published outside the Nordic 
region, more influential than national or regional references or vice versa? 
Does the institutional affiliation (government, academe, "institute sec- 
tor," private think-tank, civil society organization) of the author matter? 
Finally, the cross-national dimension enables us to examine the varied 
legitimization or authorization strategies in the five countries in depth 
and, by means of comparison, identify nation-specific patterns in the 
policy process. 

Eyal (2019) explains Luhmanns conceptualization of authority, valid- 
ity, and legitimacy of expert knowledge, juxtaposing the systems- 
theoretical approach against Jürgen Habermas’ work on the legitimacy 
crisis. In concert with Luhmann, Eyal uses “validation” in the sense of 
defensibility to "reassure people that if they would bother to check, they 
would find that the particular decision was rationally taken and justified, 
so no need to bother!" (Eyal, 2019, p. 88). For Luhmann, the insistence 
on validation is ultimately a "functionally necessary deception" that saves 
time and prevents discord, here given that every fact may also be inter- 
preted differently. 

Strikingly, the validity issue has become key at a time when informa- 
tion is openly and abundantly available and when expertise has become 
democratized, enabling users and other lay persons to participate in the 
production of evidence and render each and every piece of evidence 
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contestable. In fact, the two prominent trends in “modern governance 
practices" are, according to Krick et al. (2019, p. 927), a trend toward 
scientization and a participatory turn. The first trend is discernible in the 
composition of government-appointed advisory committees (a growing 
number of researchers), citation patterns (reference to studies, technical 
reports, and academic publications), and epistemic language (reference to 
"evidence," “knowledge,” “data,” and “research”. The second trend 
implies greater public engagement in agenda setting as a result of open 
access to information, calling into question the corporatist or representa- 
tive model of democracy (see Rommetvedt, 2017; Rommetvedt et al., 
2012). Other scholars (Stehr & Grundmann, 2011) have labeled the sec- 
ond trend as a pluralization of expertise. These two trends—scientization 
and pluralization of expertise—have triggered a third trend that has only 
recently been discussed (see Lubienski, 2019): a surplus of evidence. 
Taken together, all three trends account for the fact that references, that 
is, the sources of information used to validate the evidence, have gained 
authoritative status. 

Arguably, referencing other texts as an instrument of validation of 
one’s assertions has become an object of intense scrutiny, including in 
bibliometric network analyses. The pressure to disclose the sources of 
information that were used to produce evidence is discernible in the ever- 
increasing number of references listed in Green Papers. The Green Papers 
of the Norwegian Official Commissions (NOUs; Norges offentlige utred- 
ninger) and the White Papers of the Ministry of Education and Research 
of Norway make for good cases for demonstrating the trend over time. 
The relevant papers of the 1996 School Reform in Norway made only 
sparse use of references, many of which were either embedded in the text 
or listed as footnotes. Twenty years later, however, there were 246 refer- 
ences on average per relevant Green or White Paper for the 2020 
Curriculum Renewal Reform (Baek et al., 2018). 

Paradoxically, the proliferation of evidence-based policy planning has 
added fuel to the crisis of expertise (see Eyal, 2019). Not only has science 
become politicized and politics scientized, but science has also become 
demystified in front of everyone's eyes: 
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[T]he very discourse on expertise increases uncertainty and threatens legiti- 
macy because now the public is witness to controversies between scientists. 
(Eyal, 2019, p. 102) 


In effect, the proliferation of evidence-based policy planning has 
brought to light that evidence is considered not more, and not less, than 
a subjective assertion backed by information. The boom has generated a 
surplus of evidence to the extent that there is now the challenge of how 
to weed out evidence based on relevance and credibility criteria. 
Concretely, in the wake of complexity reduction, we are witnessing a 
hierarchization of information (very often with randomized controlled 
trials on the top and qualitative data on the bottom), rendering some 
types of evidence more relevant than others. At the same time, the disclo- 
sure of the source of information to make a case for the credibility of the 
evidence, that is, the reference, has become as important, if not more so, 
than the information itself. In fact, the legitimacy of the assertion rests in 
great part on the source of the information itself. For example, a reference 
here and there to OECD studies has become a sine qua non for policy 
analysts in Europe because the OECD is seen, in the Foucauldian sense, 
as the founder of discursiveness for a very special kind of policy knowl- 
edge that ranks at the top in the hierarchy of evidence, one that operates 
with numbers and draws on international comparisons to enforce a polit- 
ical program of accountability. Ydesen (2019) has convincingly docu- 
mented the rise of the OECD as a global education governing complex 
that uses a range of policy instruments (PISA, Education at a Glance, 
country reports, etc.) to diagnose and monitor national developments 
and advance the global solutions ofa particular kind for national reforms. 

There are many reasons why OECD studies are attractive for govern- 
ment officials (see Martens & Jakobi, 2010; Niemann & Martens, 2018). 
Espeland (2015) and Gorur (2015) masterfully observe the advantages of 
numbers over complex narratives because one may attach one’s own nar- 
ratives to numbers. What is especially appealing to policy actors are 
OECD-type studies, that is, statistics, scores, ranking, and benchmarks 
based on international comparisons or on comparisons over time. Novoa 
and Yariv-Mashal dissect the politics of international comparison and 
examine how: 
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[T]his ongoing collection, production and publication of surveys leads to 
an ‘instant democracy’, a regime of urgency that provokes a permanent 
need for self-justification. (2003, p. 427) 


Espeland (2015, p. 56) explains the dual process of simplification and 
elaboration involved in using numbers. In the first step, numbers “erase 
narratives” by systematically removing the persons, institutions, or sys- 
tems being evaluated by the indicator and the researcher doing the evalu- 
ation. This technology of simplification stimulates narratives, or as 
Espeland astutely observes: 


If the main job of indicators is to classify, reduce, simplify, and make visible 
certain kinds of knowledge, indicators are also generative in ways we some- 
times ignore: the evoke narratives, stories about what the indicators mean, 
what their virtues or limitations are, who should use them to what effect, 
their promises, and their failings. (2015, p. 65) 


Scholars in comparative policy studies have started to explore why 
PISA and other international large-scale student assessments are so attrac- 
tive to policy actors and politicians (Addey et al., 2017; Pizmony-Levy, 
2018). A few studies focused on the “narrative evoking” phase (Espeland, 
2015, p. 65) of such studies have dissected what national governments 
interpret or project onto OECD reports or other international compara- 
tive studies based on their own policy context and agenda (Waldow & 
Steiner-Khamsi, 2019). 


Reference Societies in Comparative 
Education Research 


In addition to the governance-by-numbers argument presented above, 
for many countries, the OECD represents an attractive geo-political 
space inhabited by people in 36 high-income economies. Therefore, a 
recourse to OECD publications may be seen both as an affirmation of 
the affiliation and an acknowledgment of the OECD as a “reference soci- 
ety” (Bendix, 1978, p. 292) or rather “[transnational] reference space” 
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toward which national governments orient themselves or aspire to belong. 
In fact, in all five Nordic countries of the POLNET study, OECD pub- 
lications represent the most cited international texts (see Chap. 11). This 
may come as a surprise for a non-Nordic audience because one would 
expect competition between two dominant policy discourses in the five 
countries of the POLNET study: the Nordic reference space, which tra- 
ditionally has had a strong commitment to equity, and the OECD refer- 
ence space, which has a mission to advance economic growth. 

Thus, in comparative policy studies, the term "reference" also carries a 
spatial, geo-political, or epistemological connotation. It is used in con- 
nection with “reference society” or “reference space.” The reference as a 
validation instrument and the reference as a point of epistemological ori- 
entation both share a common feature: they position the author (in the 
case of references) or the state (in the case of reference society) in its larger 
discursive space. 

By now, there is a well-established tradition in comparative policy 
studies to draw on references as an analytical tool to situate the position- 
ality of an actor (author, institution, government) in a broader transna- 
tional, geo-political space. This body of scholarship is closely associated 
with studies on the “reference society” presented by sociologist Reinhard 
Bendix (1978, p. 292). Bendix uses the term to denote how governments 
used economic competitors and military rivals as reference societies for 
their own development. One of the examples discussed by Bendix is the 
fascination of Meiji-era Japan with the West. 

In comparative education, the term was—according to Waldow 
(2019)—first introduced by Butts (1973), associate dean and professor of 
Teachers College, Columbia University. Butts observes that the govern- 
ments of developing countries frequently used a specific educational sys- 
tem in the Global North as a model for emulation. That country’s path to 
“modernization” served government officials in the Global South as a 
reference for educational reforms in their country. It is important to bear 
in mind here that during Butts’ time, transnational networks and depen- 
dencies established during colonial times had endured into the present 
and determined in great part the choice of reference societies. Another 
noted historian and comparativist, David Phillips, first coined the term 
“cross-national policy attraction” to denote the keen interest of 
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nineteenth-century British government officials in the educational 
reforms of Germany (see Ochs & Phillips, 2002). Both Butts and Phillips 
use records of study visits and government reports as sources for their 
analyses of cross-national attraction or policy borrowing. 

Similar in the conceptual framework but different in terms of unit 
analysis, Schriewer and Martinez (2004) use bibliometric data to exam- 
ine the use of reference societies in "educational knowledge," as reflected 
in the publications of educational research journals; they wonder whether 
educational researchers in their sample of three countries draw on similar 
or different bodies of knowledge or texts. They purposefully use a time 
period of 70 years to see whether a convergence toward a single interna- 
tional canon of scientific educational knowledge, here interpreted as 
internationalization or globalization, has occurred. Concretely they 
examine the references listed in flagship educational research journals in 
three countries (Spain, Russia/Soviet Union, PR China) and code them 
in terms of the national origin of the referenced authors. Rather than 
detecting a pattern of steady internationalization toward a single body of 
internationally acclaimed authors, they notice considerable fluctuation 
regarding the space allocated to international scholarship, as measured in 
the number and type of foreign bibliographical references made in the 
journal articles of the three countries. They find that the “socio-logic” 
(particularly political developments in a given country) was a better pre- 
dictor of receptiveness toward international scholarship than an external 
logic as manifested in the ever-expanding transnational network of edu- 
cational researchers. 

In fact, the era of the greatest convergence regarding educational 
knowledge was in the 1920s and 1930s, when educational researchers in 
Spain, the Soviet Union, and China were drawn to the work of John 
Dewey. Once that brief period was over, Dewey was dropped from the 
reference list in Soviet educational journals and replaced by Nadezhda 
Krupskaya (Lenin’s wife). It is striking that against all expectations of 
globalization or international convergence theorists, educational knowl- 
edge in these three countries did not become more internationalized until 
after the mid-1980s, when all three opened their ideological boundaries 
and increased international cooperation. Even though Schriewer and 
Martinezs (2004) justification for their case selection leans on a 
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problematic notion of culture and “civilization,” the design and method- 
ology of the study is compelling and well-suited for analyzing interna- 
tional convergence/divergence processes in educational research. 

The link between the reference society and political change has also 
been well documented in comparative education research. Examples 
include two cases of a radical change in reference societies as a result of 
fundamental political changes in post-Soviet Latvia and post-socialist 
Mongolia, respectively. Silova (2006) examines the erasure of Soviet ref- 
erences and their subsequent replacement with Western European refer- 
ences. She interprets the shift from the Soviet to the Western European 
reference system as a marker for the new geo-political educational space 
that Latvia politically and economically had been aspiring to inhabit at 
the turn of the millennium. What is fascinating about this particular 
change of political allies is that it has merely affected the discursive level, 
not the practice of separate schooling. The separation of school systems, 
one for Latvian speakers and another for Russian and other ethnic speak- 
ers, continues to exist; however, segregated schools are no longer seen as 
“sites of occupation” but are now being reframed as “symbols of multicul- 
turalism.” The list of comparative policy studies on reference societies is 
too long to present in an exhaustive manner. In our own study on 
Mongolia (Steiner-Khamsi & Stolpe, 2006), we observe the discursive 
ruptures and reorientation in terms of reference spaces that accompanied 
the political changes, notably the replacement of the Communist Council 
for Mutual Economic Assistance with the Asian Development, World 
Bank, and other post-communist international aid agencies. 

Strikingly, studies on reference societies and cross-national policy 
attraction have experienced a revitalization of a special sort in recent years 
with the fast advance of international large-scale student assessments 
(ILSAs) used in many countries as a policy tool for governance by num- 
bers (see Carvalho & Costa, 2015; Volante, 2018; Waldow & Steiner- 
Khamsi, 2019). Preoccupation with what league leaders (Finland, 
Shanghai, Singapore, etc.) have “done right” has generated new momen- 
tum for policy borrowing research. Precisely at a stage in policy borrow- 
ing research when scholars have put the study of cross-national policy 
attraction to rest and instead directed their attention at the ubiquitous 
diffusion processes of global education policies in the form of “best 
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practices" or "international standards" vaguely defined, the cross-national 
dimension—and by implication the focus on the nation-state and its 
national policy actors—has regained importance in ILSA policy research. 

In the case of PISA (Programme of International Student Assessment), 
the preoccupation of national policy actors is, at least rhetorically, on how 
their own system scores compared with others and what there is to “learn” 
from the league winners, league-slippers, and league-losers, in terms of 
PISA’s twenty-first-century skills. Because policy actors often attribute 
"best practices" to particular national educational systems, the national 
level regained importance as a unit of analysis. Therefore, ILSA policy 
researchers found themselves in a position of having to bring back the 
focus on national systems, a unit of analysis criticized as “methodological 
nationalism," which, if used naively, is a cause for concern because of its 
homogenizing effects (see Giddens, 1995; Wimmer & Glick Schiller, 
2003; Robertson & Dale, 2008). 

Research on reference societies has also been refined over the past few 
years in other ways. For example, intrigued by negative media accounts 
in Germany about the PISA league leader Shanghai (during the 2012 
PISA round), Waldow scrutinizes the policy usage of "negative reference 
societies" (2016) or “counter-reference societies,” respectively. The con- 
cept of a reference or counter-reference society is based on commensura- 
bility. How do national policy actors make the educational systems of 
league winners appear to be comparable to their own educational system 
in a way that can suggest that lessons could be drawn? Vice versa, how do 
they manage to make two educational systems incommensurable and 
incomparable to avoid lesson-drawing? The disbelief or the downplaying, 
respectively, of Chinese success in ILSAs, notably in the PISA rounds of 
2012 and 2018, is comparable to earlier stereotypical accounts of Japanese 
or pan-Asian education. 

Similar to the US media accounts of A Nation at Risk (1983) in which 
American policy analyses attempted but ultimately failed to persuade 
Americans of the great benefits of the German and Japanese educational 
systems, the education systems of Beijing, Shanghai, Jiangsu, and Zhejian 
are, despite “PISA success,” hardly used as models for emulation in 
Western countries. As with the A Nation at Risk report, the common 
reaction to Chinese success reflects a “yes, but ...” attitude (see Cummings, 


2 What Is in a Reference? Theoretically Understanding the Uses... 43 


1989, p. 296): even though there is a general agreement about the out- 
standing student performance in ILSAs in Hong Kong, Japan, Korea, 
Macao, Singapore, and select cities of PR China, there are too many 
negative stereotypes associated with education in these locations to assign 
them reference or emulation status. In fact, the exaggerated statements or 
myths about “Asian education” include images of overly ambitious moth- 
ers (“tiger mothers”), excessive use of cram schools, competition and sui- 
cide among students, elitist higher education, and social inequality. More 
often than not, the educational systems in Asia are politically instrumen- 
talized as a counter-reference, that is, examples of how educational sys- 
tems should zot be developed. 

Let us now circle back to the five-country study at hand. In the Nordic 
region, there is an elephant in the room in the broader ILSA space, mak- 
ing one wonder the following: Do the other countries of the region 
(Denmark, Iceland, Norway, and Sweden) consider the educational sys- 
tem of Finland (a PISA league leader) as a reference or a counter-reference 
for educational reform in their own system, or are they indifferent toward 
lesson-drawing from Finland? The coding of the references by their coun- 
try of publication and the qualitative analysis of thematic cross references 
make it possible to examine the fascinating question of a reference society 
within the Nordic education space (see Sivesind, 2019; Chap. 12). 

Clearly, the bibliometric analyses presented in this volume demon- 
strate that OECD publications eclipse studies from Finland. Two possi- 
ble yet inconclusive interpretations lend themselves to further 
investigation: either “Finnish success” is acknowledged but rendered 
irrelevant for one’s own national context (the “yes, but ...” attitude 
explained earlier), or Finnish success is, for a variety of reasons, including 
linguistic ones, referenced via an authoritative source of information: 


OECD publications. 
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Expertise-Seeking Arrangements 
in Policy Making 


In his dissertation research, Baek (2020) coins the term “expertise-seeking 
arrangements," which captures very well the dilemma of governments in 
an era of evidence-based policy making: Where and how do they seek 
advice for policy analysis, evaluation, and formulation? Given that "the 
authority of experts is destabilized" (Eyal, 2019, p. 102) in an era in 
which scientific evidence production is easily demystified and an ever- 
increasing number of individuals, including concerned citizens and other 
laypersons, lay claim to expertise, the question of governments expertise- 
seeking arrangements is taking center stage. 

Eyal presents a typology of responses to the legitimation crisis, which 
is reproduced in Table 2.1 below. His focus is on "regulatory science" or 
the "interface between scientific research, law and policy" (see Eyal, 2019, 
p. 7£.). 

The first strategy of the state is to pretend that science is purified from 
politics by pursuing “mechanical objectivity” (Eyal, 2019, p. 115), as 
reflected in references to scores, rankings, numbers, impact evaluations, 
and quantifiable comparisons. In our case, references to OECD studies, 
evaluations, and ILSAs belong to this category. 

The second strategy of inclusion is to acknowledge that science is politi- 
cized—or as Latour has eloquently put it, "Science is not politics. It is 
politics by other means” (1984, p. 229)—and, therefore, includes 


Table 2.1 Typology of responses to the legitimation crisis 


Problem of trust 


Trust in transparent, Trust in trained 
objective, public judgment of 
procedures experts 
Problem of Technocratic Objectivity Exclusion 
extension decision-making 
Participatory Inclusion Outsourcing 


decision-making 
Source: Eyal (2019, p. 103) 
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laypersons, interest groups, and other engaged citizens into the advisory 
bodies of the government. 

The third strategy of exclusion is to decouple science from politics and 
generate "gate-keeping mechanisms designed to maintain an artificial 
scarcity of expertise" (Eyal, 2019, p. 105; see also Weingart, 2003); these 
are typically academic associations (academy of sciences) or professional 
associations that claim exclusive expertise and advise the government. 

The fourth and final strategy of outsourcing represents another func- 
tional differentiation process. It decouples science from politics to the 
extent that it delegates research and policy formulation to outside groups, 
think-tanks, or semiprivate entities. Eyal contends that the fourth strat- 
egy is oftentimes a reaction to failed attempts of the state to generate trust 
or being inclusive, which would be pursued in the first three strategies. In 
particular, the fourth strategy is often a response to the critique that the 
second strategy—the appointment of ad hoc advisory commissions or 
government-appointed expert commissions—are merely meant for 
window-dressing and rarely impact the ultimate policy decisions and for- 
mulations prepared by government officials. 

Another useful typology of "advisory system activity" has been devel- 
oped by Craft and Howlett (2013, p. 193ff.). In general, they find a trend 
toward the inclusion of nonstate actors (think-tanks, open data citizen 
engagement driven policy initiatives/web 2.0, etc.) and international 
actors (e.g., OECD, ILO, UN organizations) as policy advisors. This is in 
stark contrast to traditional advisory systems, which mainly have drawn 
on national advisory bodies, including statistical offices, strategic policy 
units within the government or government-appointed ad hoc 
commissions. 

Naturally, the changing nature of the relationship between politics and 
science has preoccupied comparative policy studies for a while. One of 
the early, more important comparative studies exploring the interpene- 
tration of the two function systems is the research project "Ihe role of 
knowledge in the construction and regulation of health and education 
policy in Europe: convergences and specificities among nations and sec- 
tors," abbreviated as Know& Pol and funded in the Sixth Framework 
Programme of the European Commission (see, e.g., Fenwick et al., 
2014). Knowledge-based regulation, which is analyzed in the Know&Pol 
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research project, has fundamentally changed the role of the state from 
one that runs schools to one that establishes learning standards and mon- 
itors learning outcomes, thereby enabling a multitude of providers, 
including businesses, to enter the school market. Over the past 20 years 
or so, the private sector has become not only a major provider of educa- 
tion, but also a key policy actor, lobbying for reforms that further restrict 
the role of the state in the education sector (Verger et al., 2017). 

As mentioned above, knowledge-based regulation has also enlarged the 
radius of individuals contributing to policy-relevant educational knowl- 
edge. An early indication of changes in knowledge production and shar- 
ing is the open-access policies that both governments and research 
councils have put in place recently. System theorists Peter Weingart and 
Justus Lentsch (2008) consider such open-access policies to be part and 
parcel of a democratization of expertise; here, the relationship between 
science and politics has experienced three distinct shifts over the past 70 
years (2008, p. 207 ff.). During the early period of scientific policy advice 
(19505-19705), the ad hoc expert commissions insisted on being autono- 
mous and independent from governments. As a corollary, their reports 
amassed foundational scientific knowledge that policy actors could or 
could not use, respectively. In a second phase, the commissions became 
increasingly politicized (1970s to 1990s) because they were charged with 
the task of producing policy-relevant scientific knowledge. In the current 
third phase, the governments in many countries have experienced a shift 
from “knowledge-based legitimacy" to “participation-based legitimacy.” 
This also applies to government-appointed ad hoc expert commissions. 
Governments are under pressure to “democratize” scientific policy advice 
by (i) providing open access to reviews and expertise, (ii) expanding the 
definition of “experts” (including nowadays, both producers and con- 
sumers), and (iii) insisting that the knowledge products are useful, that is, 
provide a clear foundation for stop/go policy decisions. 

In the five participating countries of the Nordic region, there is a wide 
array of expertise-seeking arrangements that these countries’ governments 
have put in place (see Chap. 10). The Eurydice Report (2017), Support 
Mechanisms for Evidence-Based Policy-Making in Education, is incomplete 
(data on Iceland is missing) and too imprecise to provide any useful clues 
for a categorization of expertise-seeking arrangements. For example, the 
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government-appointed expert commissions in Norway (NOUs) and 
Sweden (SOUs) that amass evidence to substantiate their evaluation of 
past reforms and their recommendations for new directions are not men- 
tioned. As documented in the OECD study on policy advisory systems 
(OECD, 2017), in all five countries, there is a commitment to evidence- 
based policy planning (which in some countries is inscribed in law), an 
extensive stakeholder review, or a "hearing" process in which draft ver- 
sions of new policy are opened up for public consultation. 

‘The type of expertise-seeking arrangement in each of the five countries 
needs to be kept in mind when interpreting the role of experts in produc- 
ing evidence for policy making. It may be useful to draw on an existing 
typology of such arrangements. For example, Weingart and Lentsch 
identify six types of commissions that provide scientific advice for policy 
making (2008, Chap. 2): (i) policy-domain-specific advisory councils, 
(ii) expert commissions for risk management, (iii) policy-specific expert 
commissions, (iv) ad hoc commissions, (v) enquete commissions, and 
(vi) sector research. 

The typology may be used to categorize the five types of expertise- 
seeking arrangements in the countries of the Nordic region. According to 
the typology of Weingart and Lentsch (2008), the government-appointed 
“official commissions” in Norway and Sweden (NOUs and SOUs, respec- 
tively), which prepare and help legitimize policy decisions, fall into the 
categorization of “ad hoc commissions.” In Denmark, the School Council, 
which was established in 2006, serves to advise the ministry on topics 
related to elementary school (see Chap. 4). In Finland, the expertise- 
seeking arrangement is multisited or hybridized, according to Holli and 
Turkka (2021). The government-appointed ad hoc commissions, which 
exist in Norway and Sweden, were abolished in 2003 and replaced with 
broad-based working groups. The representation of academics declined 
over time and constituted only 4.7% of all working group members in 
2015. At the same time, the Government of Finland pluralized the policy 
advisory system: 


[T]he policy advisory system of Finland shows signs of hybridisation, as 
the channels and organization of policy advice have pluralised and advice 
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has taken new forms. (Holli & Turkka, 2021, p. 58 [translation by 
the authors]) 


The partial externalization of policy advice is manifested in the rise of 
"state investigators" or consultants who are hired to produce government- 
commissioned reports to a general outsourcing of policy research. 
According to the authors, Finland has created a "research market," where 
the state buys the research it needs for policy preparation. In Iceland, 
finally, the composition of advisory bodies is strictly regulated in terms of 
gender and political parties to ensure an inclusive consultative process. 
According to the OECD survey on policy advisory systems, which is car- 
ried out in 17 countries, including the 5 Nordic countries studied here, 
the policy advisory system in Iceland requires that at least 40 of the mem- 
bers of ad hoc advisory commissions are female and that all political par- 
ties are represented (OECD, 2017). 

The OECD has formulated five quality standards for policy advisory 
systems: adaptability, transparency, autonomy, inclusiveness, and effec- 
tiveness. Ihe OECD 17-country study (OECD, 2017) presents a posi- 
tive assessment of the ad hoc advisory committees found in the 
Nordic region: 


Ad hoc advisory bodies [...] are often used by governments to gather 
evidence-based answers to particular questions relatively quickly. They 
often serve as a "fast track" and specialized option for governments to 
obtain advice. The Nordic countries have well-established traditions of cre- 
ating ad hoc bodies to enhance the adaptability of the system. (OECD, 
2017, p. 17) 


Implications for the Five-Country Bibliometric 
Network Analyses 


In the parallel universe of “gray literature” or technical reports, which are 
often commissioned by international organizations, there is an interest- 
ing discussion unfolding on the rapid spread of “global public goods” 
(GPGs) or global knowledge banks. GPGs include, for example, openly 
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accessible international toolkits, documents, studies and databanks, 
training modules, good practices, and global monitoring reports. Clearly, 
the GPGs are openly accessible information, nowadays often backed up 
with numbers that any local, national, or international organization may 
use to back up their production of evidence. The rapid spread of GPGs 
was addressed by a few scholars around the turn of the millennium (nota- 
bly Stone, 2000), but curiously, the discussion has not yet gained traction 
in academic debates. Hence, a brief summary of the debates, carried out 
in the context of development studies, may be in order here. 

Within development studies, the discussion is now bifurcating in at 
least two different directions. One group of authors makes the argument 
for more funding for GPGs produced by a more diverse body of research- 
ers (based in the Global North and the Global South) and another group 
critically examines the uptake of GPGs for national policy making at the 
national level (see Vasquez Cuevas, 2020). 

In some countries, a wide range of propositions have been made about 
how to remedy the shortfalls related to global agenda setting, channeling 
of aid, and GPGs (see Scháferhoff & Burnett, 2016). Some suggestions 
entail more funding at the global level, whereas others notice that the 
production of GPGs is mostly done by a few global actors (OECD, the 
World Bank, the UN system) at the expense of national research institu- 
tions outside of North America, Europe, and Australia. This applies in 
particular to think-tanks, research institutions, and universities in the 
Global South, whose knowledge products are rarely taken up at the global 
level. Examples of more funding to the Global South include Oxfams 
early suggestion to eliminate one-size-fits-all benchmarking processes and 
dedicate three grants for capacity building to recipient governments and 
civil society organizations of low- and lower-middle-income countries 
(Oxfam, 2010). 

For a while, the question arose whether the World Bank, UNESCO 
institutions, UNICEF, Global Partnership for Education, or other inter- 
national organizations should earmark funds for research capacity build- 
ing and policy analysis. One of the early suggestions was to increase 
funding for the global and regional agencies of UNESCO and UNICEF 
to advance cross-country sharing of knowledge on education and devel- 
opment. In addition to statistics, the UN organizations would use the 
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funds to disseminate knowledge derived from research and from global 
sharing of experience. Others found the World Bank to be ideally suited 
for helping expand research funding and activities given its commitment 
to evidence-based policy making; they recommended that researchers at 
the World Bank would work more closely with other staff for country- 
level policy advice (Clemens & Kremer, 2016). Unsurprisingly, the lively 
debate in development studies is on whose knowledge is made publicly 
available at the global level and whose knowledge is confined to the 
national boundaries of its producers. 

In OECD countries, the debate over the asymmetry of global knowl- 
edge production and update seems to center more on the language of 
publication (English vs. all other languages) rather than on the center/ 
periphery differentiation in an unequal world system. Some countries in 
the Nordic region (Sweden and Norway in particular) require that all 
public documents be made openly accessible. The transparency standard 
of policy advisory systems, which are forcefully promoted by the OECD 
(2015, 2017), is also practiced in the other three Nordic countries. 

The open-access policies and practices have both enabled and exacer- 
bated the “participatory turn" or the “pluralization of expertise" (see 
Krick et al., 2019), which has been explained above. The decline in cor- 
poratism or interest group representation in policy advisory systems is 
but one of the manifestations of this trend. Another manifestation is the 
surplus of evidence. 

In the US context, Lubienski (2019) contends that there is not a scar- 
city but rather a “surplus of evidence.” In such a “marketplace of ideas,” 
there is ample opportunity for new, nonstate actors—specifically the pri- 
vate sector—to serve as intermediaries between research production and 
policy making: 


Into the chasm between research production and policy-making, we are 
seeing the entrance of new actors— networks of intermediaries—that seek 
to collect, interpret, package, and promote evidence for policymakers to 
use in forming their decisions. (Lubienski, 2019, p. 70) 


Indeed, two decades after neoliberal calls for less politics and more 
scientific rationality in the policy process, we are now entering a new 
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phase in policy making: the stage of surplus of evidence in which calls for 
“actionable research," “policy-relevant research,” or “what works" studies 
are being heard. At the 2015 Public Governance Ministerial Meeting in 
Helsinki, the ministers from OECD countries agreed (OECD, 2015, 
p. 3) that evidence alone is not sufficient. Evidence needs to be policy 
relevant, robust, and comparable: 


We acknowledge the importance of evidence as a critical underpinning of 
public policies and recognize the need for a continuous effort to develop 
policy-relevant evidence on government performance that is robust and 
comparable. “What Works” initiatives are an example of how to ensure 
systematic assessment and leverage the stock of information on good prac- 
tices available at the international level on policy impact. (OECD, 
2015, p. 3) 


In the Nordic policy context, the OECD plays a major role as a trans- 
national policy advisor and standard setter. During the first phase of 
evidence-based policy planning, the OECD advanced the notion of 
autonomous policy advisor systems that produce, independent of the 
state, evidence. In today's stage of evidence overproduction, the OECD 
offers itself as an interpreter of evidence by selecting from the market- 
place of ideas those that are actionable and in line with the broader politi- 
cal program of accountability. 

The proliferation of GPGs and the overproduction of evidence make it 
pressing to investigate the changing role of government-appointed advi- 
sory commissions in the production and interpretation of evidence. A 
host of research questions open up once we acknowledge that the presen- 
tation of “facts” (information) and transformation of these facts into evi- 
dence rests on a subjective selection process that reflects the frame of 
reference or broader discursive orientation of the author. 

Based on the elaborations presented above, we have several research 
questions that lend themselves to a systems-theoretical preoccupation 
with evidence-based policy planning: First, what sources of information 
do government-appointed advisory panels consider credible and, there- 
fore, select to reduce uncertainty and generate trust? Second, what kind 
of hierarchization of evidence do government-appointed advisory panels 
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generate to reduce complexity? Third, bearing in mind the three most 
common types of externalization, what type of externalization do the 
cited texts represent: reference to (i) tradition or values (e.g., Nordic value 
of equity), (ii) organization (e.g., reference to laws and regulations), or 
(iii) scientific rationality (e.g., studies and evaluations). Fourth, in which 
broader epistemic community, or rather political reference space, do the 
authors situate themselves? Finally, given the academization of 
government-appointed advisory commissions in some countries 
(Christensen & Hesstvedt, 2018), as a result of which the number of 
academics serving as members increased at the expense of interest group 
representatives, the phenomenon of structural coupling between science 
and politics (Steiner-Khamsi et al., 2019; Weingart, 2003; Stehr & 
Grundmann, 2011; see also Chap. 10) offers itself as an object of empiri- 
cal scrutiny. 

This chapter has attempted to demonstrate how a bibliometric net- 
work analysis may be used as a method of inquiry for understanding 
legitimization processes in evidence-based policy making. Drawing our 
attention to how uncertainty is reduced and trust in evidence is created is 
essential in an era in which there is a surplus and, by implication, com- 
peting notions of evidence. A bibliometric investigation of the references 
in a text provides important clues about the (i) selection of sources of 
information, (ii) hierarchization of evidence, (iii) and type of externaliza- 
tion made by an author to leverage authority for the claims made in the 
text. A network analysis of the references further helps complicate the 
findings. In fact, the focus on relations brings to light that different 
authors use the same references for different purposes, that is, one and the 
same reference may show up in, and bridge, two different knowledge 
networks. Perhaps needless to reiterate, the long list of fascinating research 
questions, which are presented in a nonexhaustive manner in this chap- 
ter, gains additional attraction when investigated across the five different 
national contexts within the Nordic region. 
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Knowledge: A Methodological Note 
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The intention of this research project has been to examine how policy- 
makers mobilize evidence to advance educational reforms. Specifically, we 
sought to inspect how policymakers in five countries—Denmark, Finland, 
Iceland, Norway, and Sweden—link different types of evidence to policy, 
and to explore differences and commonalities between these countries. 
This set of objectives required a careful look into the architecture of policy 
knowledge, which includes the visible links between policy documents 
and other knowledge artifacts, such as articles, chapters, books, reports, 
and statistical analyses. In what follows, we describe the method behind 
this research project and the procedures we employed in each of the chap- 
ters. We detail important decisions about our methodological approach 
for analyzing and presenting data from policy documents. 
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Theoretical and Methodological Inspirations 


Two lines of research inspired the research design: (a) social network anal- 
ysis and (b) sociology of knowledge. In this section, we discuss the prem- 
ise of each literature and the main concepts. We posit that a synthesis of 
these two literatures could offer a new way to examine how policymakers 
mobilize evidence to advance educational reforms. 

The first is social network analysis (SNA), which is a broad research 
paradigm that includes theory, substance, and methodology. The basic 
definition of a social network is “a finite set or sets of actors and the rela- 
tion or relations defined on them" (Wasserman & Faust, 1994, p. 20). In 
other words, a network is a set of socially relevant nodes that are possibly 
connected by one or more relations (Marin & Wellman, 2011). While 
some actors are connected, other actors might be disconcerted or isolated 
from each other. The key premise of SNA is that relationships between 
actors influence outcomes (e.g., attitudes and behaviors) beyond the 
actors characteristics alone (Valente, 2010). Further relationships 
between actors determine in part what happens to a group of actors as a 
whole. Therefore, social network scholars examine the structure that 
emerges from these social patterns with the objective of understanding 
the ways in which this structure contributes to specific outcomes. Because 
actors and relationships are fundamental components in SNA, we now 
turn to defining these concepts. 

Actors (or nodes) are discrete units or groups. Examples of actors are 
students in a classroom, schools within an educational system, non- 
governmental organizations (NGOs) in a given field, or nation-states in 
the world system. Most social network studies focus on collections of 
actors that are all the same type (e.g., schools in the same education sys- 
tem). However, some studies include actors from different sets (e.g., 
schools and NGOs that support them). The former is often called a one- 
mode network, whereas the latter is often called a two-mode network. 
Actors have attributes—or characteristics—that describe and distinguish 
them. For example, students could be described by their sex/gender, age, 
grade, and socio-economic background; schools could be described by 
their location (rural/urban), selectivity, and extent of important resources. 
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Relationships (or edges) are the ties that connect actors. The defining 
feature of a relationship is that it establishes a linkage between two actors. 
Relationships among actors can be of different kinds, and each type facil- 
itates a corresponding network. Borgatti and Ofem (2010) offer a useful 
typology of relations studied in SNA that divides relations into five types: 
(a) similarities include spatial and temporal proximities, co-membership 
in groups or events, and sharing an attribute; (b) social relations are ties 
such as kinship and friendship; (c) mental relations are perceptions of 
and attitudes toward others; (d) interactions are discrete events that can 
be tallied over a period; and (e) flows are interactions that are transmitted. 

SNA posits that an actor's position in each network shapes the oppor- 
tunities and constraints that the actor will encounter. This perspective is 
distinctive from traditional social science, which focuses on the charac- 
teristics of actors as predictors of different outcomes. In traditional social 
science, we might explain differences in the performance of individuals or 
groups by certain qualities or characteristics. In contrast, SNA considers 
the web of relationships in which individuals or groups are embedded. 

In this research project, actors/nodes are documents produced and 
used in a given policy space.' Specifically, we operationalize the architec- 
ture of policy knowledge as a two-mode network that consists of source 
documents and reference documents. Source documents include a set of 
white papers (WPs) and green papers (GPs); reference documents include 
other artifacts referenced in the source documents. We posit that policy 
documents lend themselves as a strategic site to examine how policymak- 
ers draw on evidence and justify their political decisions. We describe 
each document with the following attributes: year of publication, type of 
publication (e.g., articles, chapters, books, reports, and statistical analy- 
ses), author (either individual or institutional), publisher, and place of 
origin (e.g., domestic, regional, international). We focus on relationships/ 
edges between policy documents and other knowledge artifacts. 
Specifically, we examine citations as particularly important connections 
between policy documents and evidence. We assume that authors of pol- 
icy documents mobilize evidence—and thus cite knowledge artifacts—to 
persuade audiences of the legitimacy of a policy statement. In other 
words, references/citations are interactions through which authority 
flows from policy documents to evidence. 
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The second line of research that informs this research project is the 
sociology of knowledge, which explores the production of knowledge 
(any knowledge) as a social activity. For example, recent work in the soci- 
ology of knowledge demonstrates a direct link between scientific collabo- 
ration networks and the structure of ideas. Moody (2004) analyzes 
patterns of co-authorship in all English journal articles listed in sociologi- 
cal abstracts that were published between 1963 and 1999. The results 
show that research specialty and methodology shape participation in the 
sociology collaboration network. Pizmony-Levy (2016) analyzed joint 
membership patterns in the Comparative and International Education 
Society (CIES) Special Interest Groups (SIGs), arguing that membership 
in these groups indicates a commitment to the fields of interest. The 
research suggests that thematic SIGs (e.g., Globalization and Education 
and Higher Education) are more central than regional SIGs. 

Specifically, we draw on bibliometric analysis, which identifies promi- 
nent authors, documents, and journals within a scientific community 
(Börner et al., 2003). Basic bibliometric analysis uses descriptive statistics 
to document trends in topics and research approaches used by scholars. 
More advanced bibliometric analysis uses network analysis to provide a 
deeper and more comprehensive view of relational and structural features 
of a given corpus of knowledge. For example, Menashy and Read (2016) 
examined the references in World Bank publications to identify the dis- 
ciplinary foundation and the geographic representation of Bank knowl- 
edge on the theme of private sector engagement in education. Verger 
et al. (2019) looked at the references in the education privatization litera- 
ture to explore the bibliographic coupling of academic and international 
agency's body of knowledge on education privatization. 

Citations—also known as bibliographic references—are the building 
blocks of any bibliometric analysis. Martyn (1975) argues that biblio- 
graphic references “expressly state a connection between two documents, 
one which cites and the other which is cited” (p. 290). Merton (1973) 
asserts that citations are designed to “prove the historical lineage of 
knowledge and to guide readers of new work to sources they may want to 
check or draw upon themselves” (p. VI). Indeed, scientific tradition 
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requires that scientists, when reporting their own research, refer to earlier 
works that relate to their research (Nicolaisen, 2007). Each bibliographic 
reference is an inscription (Latour & Woolgar, 1986, pp. 45—53) describ- 
ing a certain text by a standardized code that includes author name, title, 
journal name, publisher, year of publication, and page numbers. The 
impact of a publication is often gauged by the number of times it has 
been cited by other authors. 

In this research project, we combine SNA and bibliometric analysis to 
examine the architecture of policy knowledge in five countries. We inves- 
tigate the extent to which policy documents cite reference documents 
included in our database. Citation networks, such as the one we study, 
are more of a sociocultural network in that authors of policy documents 
may cite other authors they have never met or could not possibly have 
met (White, 2011). Whereas bibliometric analysis of scientific papers 
often uses existing databases of scientific and scholarly research (e.g., Web 
of Science), there are no similar databases for policy documents. Therefore, 
in the next section, we describe the data and methods we adopt for this 
research project. 


Constructing Networks 


Researchers have been creative in obtaining data on social networks from 
diverse sources. In addition to using surveys and questionnaires, scholars 
have used archival sources and other documents extensively (Marsden, 
1990). Interested in international student mobility, for example, Shields 
(2013) assembled information from country reports to the UNESCO 
Institute of Statistics on incoming students and their country of origin. 
Addressing questions about the system of "reference societies" in the con- 
text of education reform, Kessler and Pizmony-Levy (2020) extracted 
information from news stories published following the release of OECD/ 
PISA results in 23 countries. 

To understand the architecture of policy knowledge, we examined a 
sample of official policy documents from each of the five countries par- 
ticipating in the study. Each national team has identified a set of key 
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Table 3.1 List of reforms by country 


Country Years Title 
Denmark 2013 The Public School Reform 
Finland 2014 National Core Curriculum for Basic Education 


Iceland 2014/2018 Renewal of the Icelandic National Curriculum Guide for 
Compulsory Schools with Subjects Areas 

Norway 2016/2020 Renewal of the Knowledge Promotion Reform 

Sweden 2015/2018 A Gathering for School—National Strategy for 
Knowledge and Equivalence 


policy documents that reflected official/state policy knowledge that the 
government had used in preparation for the most recent school reform in 
each country. Table 3.1 presents the titles of the reforms in all five coun- 
tries. We first selected WPs that outlined the policy proposal in each 
country. The purpose of WPs is to launch a debate with stakeholders, 
including the public, unions, civil society, parliament, and the govern- 
ment. We then selected GPs that were explicitly cited in the WPs. GPs are 
written by government-appointed expert commissions to stimulate dis- 
cussion on given topics; they often reflect insights from multiple sources 
and relevant parties. In countries where WPs and GPs are not a compul- 
sory part of the institutionalized education policy process, each national 
team carefully identified official policy documents that are functionally 
equivalent to the WPs and GPs, respectively, in their policymaking con- 
texts (see Chap. 9 for comparative discussion on the reform and policy- 
making contexts of the five Nordic countries). In this book, we use the 
terms “WP” and “GP” to refer to white papers and green papers as well 
as their functional equivalents. Our final sample of policy documents 
includes 8 WPs and 30 GPs (see Table 3.2). 

There are strengths and weaknesses to this sampling strategy. On the 
one hand, this strategy ensures the comparability of concepts and 
results in the project. By drawing on policy documents published in 
the context of a recent education reform, we can also assess the promi- 
nence of similar international knowledge artifacts, such as OECD/ 
PISA reports. On the other hand, our focus on official policy docu- 
ments excludes texts produced by other stakeholders that participate 
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Table 3.2 Policy documents (source), by country 


Country Type Year 


Denmark WP 


GP 


GP 


GP 


GP 


Finland WP 
GP 


GP 


GP 
GP 


GP 


GP 


GP 


GP 


GP 


2012 


2011 


2011 


2011 


2012 


2012 
2002 


2010 


2010 
2010 


2011 


2012 


2012 


2012 


2012 


Title 


Gør en god skole bedre—et fagligt loft af folkeskolen 
[Make a Good School Better—Improving the Academic 
Level of the Public School] 

Undervisningsdifferentiering som baerende paedagogisk pricip 
[Differentiated Teaching as a Core Pedagogical Principle] 

Ledelse af folkeskolerne—vilkar og former for skoleledelse 
[Leadership in the Public Schools—Conditions and Forms 
of School Management] 

Beretning om Evaluering og Kvalitetsudvikling af 
Folkeskolen 2011 [Report on Evaluation and Quality 
Development of the Public School 2011] 

Beretning om Evaluering og Kvalitetsudvikling af 
Folkeskolen 2012 [Report on Evaluation and Quality 
Development of the Public School 2012] 

Tulevaisuuden perusopetus [Future Basic Education] 

Opinto-ohjauksen tila 2002—Opinto-ohjauksen arviointi 
perusopetuksessa, lukiossa ja ammatillisessa 
koulutuksessa seká koulutuksen siirtymávaiheissa 
[Evaluation of Student Counseling in Basic Education, 
Upper Secondary Schools and Vocational Education and 
in Transition Phases of Education] 

Perusopetus 2020—yleiset valtakunnalliset tavoitteet ja 
tuntijako [Basic Education 2020: Common National Aims 
and Division of Teaching Hours] 

Opettajat Suomessa 2010 [Teachers in Finland 2010] 

Esi- ja perusopetuksen opetussuunnitelmajárjestelmán 
toimivuus [Evaluation on the Curriculum of Pre-School 
and Primary Education] 

Liikunnan oppimistulosten seuranta-arviointi 
perusopetuksessa 2010. Koulutuksen seurantaraportit 
2011:4 [Evaluation of Learning Results in Physical 
Education 2010. Educational Evaluations 201 1:4] 

Onko laskutaito laskussa? Matematiikan oppimistulokset 
peruskoulun pááttóvaiheessa 2011 [Are Mathematical 
Skills in Decline? Math Learning Results at the End of 
Basic Education in 2011] 

Aihekokonaisuuksien tavoitteiden toteutumisen seuranta- 
arviointi 2010 [Evaluation of Achievement of Over- 
Arching Education Goals 2010] 

Luonnontieteiden seuranta-arviointi [Evaluation of Natural 
Sciences] 

Historian ja yhteiskuntaopin oppimistulokset 
perusopetuksen pááttóvaiheessa 2011 [Evaluation of 
Learning Results in History and Social Studies at the End 
of Basic Education 2011] 


(continued) 
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Country 


Iceland 


Norway 
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(continued) 


Type Year Title 


WP 2014 Hvítbók um umbeetur í menntun [White Paper on 
Education Reform] 

WP 2017 Education for All in Iceland. External Audit of the Icelandic 
System for Inclusive Education 

GP 2014 Review of Policies to Improve the Effectiveness of Resource 
Use in Schools: Country Background Report, Iceland. 

GP 2015 Mat á framkvæmd stefnu um skóla án aðgreiningar. Skýrsla 
starfshóps [Evaluation of the Implementation of the 
Strategy of Inclusive Education. Report of a Workgroup] 

WP 2016 St.meld.nr. 28 (2015-2016): Fag—Fordypning—Forstáelse— 
En fornyelse av Kunnskapsløftet [Report No. 28 to the 
Parliament: Subjects, In-Depth Learning—Understanding. 
A Renewal of the Knowledge Promotion Reform] 

WP 2017 St.meld.nr. 21 (2016-2017): Laerelyst—tidlig innsats og 
kvalitet i skolen [Report No. 21 to the Parliament: Eager 
to Learn—Early Intervention and Quality in Schools] 

GP 2003 NOU 2003:16 I første rekke. Forsterket kvalitet i en 
grunnopplæring for alle [In the First Row. Increased 
Quality Within a Basic Education System for Everyone] 

GP 2007 NOU 2007:6 Formal for framtida. Formal for barnehagen 
og opplæringen [Objects Clause for Kindergarten and 
Primary and Secondary Education] 

GP 2009 NOU 2009:18. Rett til læring [Students' Rights to Learning] 

GP 2010 NOU 2010:7 Mangfold og mestring- Flerspráklige barn, 
unge og vaksne i opplaeringssystemet [Diversity and 
Mastering. Multilingual Children, Young People and 
Adults in the Education System] 

GP 2014 NOU 2014: Elevenes læring | fremtidens skole: Et 
kunnskapgrunnlag [Pupils’ Learning in the School of the 
Future. A Knowledge Base] 

GP 2015 NOU 2015:8 Fremtidens skole. Fornyelse av fag og 
kompetanser [The School of the Future. Renewal of 
Subjects and Competences] 

GP 2015 NOU 2015:2 A hore til. Virkemidler for et trygt psykososialt 
skolemiljø [About Belonging and a Safe Psycho-Social 
School Environment] 

GP 2016 NOU 2016:14 Mer à hente—Bedre læring for elever med 
stort laeringspotensiale [More to Gain—Better Learning 
for Students with Higher Learning Potential] 


(continued) 
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Country Type Year Title 


Sweden WP 


GP 


GP 


GP 


GP 


GP 


GP 


GP 


GP 


2018 


2008 


2013 


2015 


2016 


2016 


2016 


2017 


2017 


Prop. 2017/18:182 Samling fór skolan [White Paper 
2017/18:182 Gathering for School] 

SOU 2008:52 Legitimation och skárpta behórighetsregler 
[Certification and Stricter Eligibility Rules] 

SOU 2013:56 Friskolorna i samhállet [The Independent 
Schools in Society] 

SOU 2015:22 Rektorn och styrkedjan, Betánkande av 
utredningen om rektorernas arbetssituation inom 
skolvásendet [The Principal and the Steering Chain. 
Report from the Commission of Inquiry into the 
Principal's Work Situation in the School System] 

SOU 2016:59 Pa goda grunder—en átgárdsgaranti for 
lásning, skrivning och matematik. Betánkande av 
Utredningen om en Lása-skriva-rákna-garanti [On Good 
Grounds—An Action Guarantee for Reading, Writing and 
Math. Report of the Inquiry Into a 
Read-Write-Count-Guarantee] 

SOU 2016:94 Saknad! Uppmárksamma elevers fránvaro och 
agera. Betánkande av Att vánda fránvaro till nárvaro—en 
utredning om problematisk elevfránvaro [Missing! Pay 
Attention to the Students' Absence and Take Action. 
Report of Turning Absenteeism to Attendance—An 
Investigation into Problematic Student Absenteeism] 

SOU 2016:66. Det stámmer! Ókad transparens och mer lika 
villkor [That is Correct! Increased Transparency and More 
Equal Conditions] 

SOU 2017:35 Samling för skolan—Nationell strategi för 
kunskap och likvärdighet. Slutbetänkande av 2015 års 
skolkommission [Gathering for School—National Strategy 
for Knowledge and Equality. Final Report from the 2015 
School Commission] 

SOU 2017:51 Utbildning, undervisning och ledning— 
reformvárd till stöd för en bättre skola [Education, 
Teaching and Management—Reform Care in Support of a 
Better School] 
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in the policymaking process. Therefore, we have a limited perspective 
on the kind of evidence that is mobilized in the policymaking process. 
Future research could address this limitation by incorporating policy 
documents produced by stakeholders such as labor unions, think 
tanks, and civil society organizations. 

Once we settled on the sample of policy documents for each country, 
we implemented a standardized procedure for coding individual refer- 
ences from each document. We trained and supervised national research 
teams from the five countries participating in the study. Each team 
included two to three members; all of them had sufficient fluency and 
familiarity with education politics and policy to read and code policy 
documents. The research team from each participating country was 
responsible for their country's data entry. All research teams followed a 
detailed protocol and used an Excel spreadsheet to enter the data. Data 
entry began by extracting all the items in the bibliography or reference 
list. That is, we coded references and not in-text citations. Research teams 
coded every reference in each source document (WPs and GPs) as the 
unit of analysis; they noted the content of the reference (e.g., author, year, 
title, publisher, type, and location). References were categorized as one of 
the five document types (reports, books, journal articles, government- 
published documents, and others) as well as one of the three location 
groups (domestic, regional/Nordic, and international). As international 
collaborations and multinational co-authorship increase, it has become 
more challenging to classify the location of a publication. In this project, 
the location was coded based on the location of the publisher. Figure 3.1 
shows the first page of the reference section for a GP in Norway: NOU 
2015:8 Fremtidens skole (Ihe School of the Future). The page includes 
23 citations; each of them was entered as a unit/record in the database. In 
addition to coding the relationship between the GP and the references, 
we also coded the references’ attributes. For example, we coded back- 
ground information for the following citations: 
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Aase, L. (2005) «Skolefagenes ulike formal — dan- 
ning og nytte». In L. Borhaug, A.-B. Fenner 
and L. Aase (red.) Fagenes begrunnelser. Sko- 
lens fag og arbeidsmáter(red.) danningsperspek- 
tiv. Bergen: Fagbokforlaget. 

Aasen, A. J., K. L. Berge, A. Skaftun and T. F. 
Hoem (2014) Lese- og skrivekompetanse i frem- 
tidens samfunn. Innspill til Fremtidens skole, 
Ludvigsen-utvalget fra Lesesenteret og Skrive- 
senteret. 

Aasen, P, J. Moller, E. Rye, E. Ottesen, T. S. Proitz 
and F Hertzberg (2012) Kunnskapsloftet som 
styringsreform — et loft eller et løfte? Forvalt- 
ningsnivåenes og institusjonenes rolle i imple- 
menteringen av reformen. NIFU Rapport 20/ 
2012. Oslo: NIFU. 

Alderson, C. and D. Wall (1993) “Does Washback 
exist?“ In Applied Linguistics, 14. 

Alexander. P. A. (2012) “Reading into the Future: 
Competence for the 21st Century”. In Educati- 
onal Psychologist 47 (4). 

Anderson, L.W. and D.R. Krathwohl (eds.) (2001) 
A taxonomy for learning, teaching, and asses- 
sing: A revision of Bloom’s taxonomy of educati- 
onal objectives. Allyn and Bacon. 

Autor, D., E. Levy and R. Murnane (2003) "The 
skill content of recent technological change: 
An empirical exploration.“ In The Quarterly 
Journal of Economics. Volume 118/4. Oxford 
University Press. 

Baird, J., T. N. Hopfenbeck, P. Newton, G. Stobart 
and A. T. Steen-Utheim (2014) State of the Field 
Review. Assessment and Learning. Oslo: Norsk 
kunnskapssenter for utdanning, 13/4697. 
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Bandura, A. (2012) «On the functional properties 
of perceived self-efficacy revisited». In Journal 
of Management, 38 (1). 

Barnes, likestillings- og inkluderingssdepartemen- 
tet (2013) Meld. St. 6 (2012-2013) En helhetlig 
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Fig. 3.1 Green paper from Norway, NOU 2015:8 Fremtidens skole. Fornyelse av 
fag og kompetanser [The School of the Future. Renewal of Subjects and 
Competences] 
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The content of the reference was entered as it was listed in the source 
documents; however, when any errors were suspected or observed, national 
teams checked the cited document to address the issues. Ihroughout the 
data entry and coding process, we communicated closely with the national 
teams to ensure validity and reliability across the countries. 

After the initial data entry, the national research teams prepared the 
data for network analysis. First, they assigned a unique identification 
number to each source document and reference document. The identifi- 
cation number included three digits country (ISO-UN code) and three- 
four digits for the specific document. Second, they combined variants of 
the same text under one identification number. For example, they merged 
under one identification number references to the original and translated 
versions of John Hatties book Visible Learning: A Synthesis of Over 800 
Meta-analyses Relating to Achievement (2008). Furthermore, one identifi- 
cation number was assigned for different editions of the same document 
if other contents of the reference, such as authors and publishers, remained 
the same. For serial publications such as OECD's Education at a Glance, 
only publications with the same title and author(s), published in the 
same year, were assigned the same identification number. 

The national research teams sent the data to the technical team at 
Teachers College, Columbia University for further review and cleaning. 
The main purpose of this cleaning was to ensure that all information in the 
database was ready for SNA. After multiple rounds of data cleaning, we 
finalized the database and constructed the network matrices for the analysis. 

Finally, we produced five sets (one for each country) of a social net- 
work file and an attributes file. The network file includes a two-mode 
matrix, with source documents (WPs or GPs) in columns and reference 
documents in rows. A cell in the matrix is coded one (1) if the source 
document cites the reference document and coded zero (0) otherwise. 
The attributes file includes background variables describing the docu- 
ments (see above). Figure 3.2 illustrates this process. 

In the figure, circles mark nodes/actors and lines mark reference rela- 
tionships. White Paper #1 cites two Green Papers (#1 and #2) and one 
reference document. Green Paper #1 cites two reference documents (#1 
and #2). Green Paper #2 cites three reference documents (#1, #3, and 
#4). Reference document #1 has an in-degree of three; that is, three 
source documents cite this reference document. All other reference 
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Reference Reference Reference Reference 
"8 H2 #3 #4 


Fig. 3.2 Illustrative example of a policy knowledge network 


documents have an in-degree of one. The following matrix represents the 
relationships in the figure as a two-mode network: 


WP #1 GP #1 GP #2 
Reference 1 1 1 1 
Reference 2 0 1 
Reference 3 0 0 1 
Reference 4 0 0 1 


Analyzing Networks: Exploring 
the Architecture of Policy Knowledge 


In this book, the authors report data from policy knowledge networks in 
five Nordic countries: Denmark, Finland, Iceland, Norway, and Sweden. 
Our analytical strategy included two steps. First, we calculated measures 
of centrality (degree) and centralization (density). Second, we produced 
visuals or maps of the networks. We conducted all analyses with 
UCINET 6.708 (Borgatti et al., 2002). UCINET is a comprehensive 
package for the analysis of social network data; most importantly, it can 
handle large networks. 
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At the micro-level, we calculated the degree centrality of reference 
documents. Degree centrality is the simplest centrality measure to com- 
pute; it is simply a count of how many connections an actor has with 
other actors (Wasserman & Faust, 1994). In this research, the degree 
centrality is equal to the number of source documents that cite a given 
reference document. For example, if WP 1 and GP 2 mention the same 
reference document, then the degree centrality for that reference docu- 
ment is equal to two. Following past research, we assume that documents 
with a higher number of citations are prominent or important in the 
context of the policy process. 

At the meso-level, we examined descriptive statistics of the degree 
centrality (i.e., minimum and maximum, average, and standard devia- 
tion). Using these simple indicators, we assessed variability across the 
reference documents (Wasserman & Faust, 1994). Low variability sug- 
gests that reference documents are homogeneous in their degree central- 
ity and structural position in the network. High variability suggests that 
reference documents are more heterogeneous in their degree centrality 
and structural position in the network. 

At the macro-level, we examined the density of the entire network. 
Density is the ratio of the number of links to the maximum possible 
number of links. Higher density means that policy documents (i.e., WPs 
and GPs) draw on similar sources for evidence. 

Finally, we used NetDraw to visualize the relationships between source 
documents and reference documents in the dataset. All figures use a 
Multidimensional Scaling (MDS) layout with node repulsion and equal 
edge length bias (Borgatti et al., 2002). This approach puts two docu- 
ments (nodes) closer together if they are more similar (in terms of their 
connections to other nodes). The distances between documents and the 
direction (or location) are interpretable. 


Limitations 


Our study has three limitations that readers should consider. First, we 
opted to code whether a source document cites a reference document 
(binary variable yes/no) and to ignore the number of times a source docu- 
ment cites a reference document. This means that our analysis does not 
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distinguish between documents cited only once and documents cited in a 
variety of different ways and places over the course of the source document. 
Second, our coding protocol did not address the context in which source 
documents cited references. Like bibliographical analysis of books and arti- 
cles, we expect that authors of policy documents can use references con- 
structively or critically (Mayrl & Wilson, 2020). 'Third, our coding protocol 
focuses on explicit references. Thus, it overlooks implicit references to cul- 
ture, norms, and values (Steiner-Khamsi & Waldow, 2018; Waldow, 2017) 
that might be included in the text. Lastly, we did not carry out any multi- 
variate analyses or statistical network modeling at this stage of the project. 
Despite these limitations, the database and analytic approaches utilized in 
this research project provide a unique opportunity to examine and map 
how policymakers mobilize evidence to advance educational reforms. 


Conclusion 


In this methodological note, we presented the theoretical and method- 
ological inspirations behind this research project. We defined key con- 
cepts—network, actors/nodes, relationships/edges, and bibliometric 
analysis—and described the process that led to the database. The follow- 
ing chapters draw on the database and analysis we described above. Some 
chapters also draw on additional methodological approaches such as 
semantic and content analysis of policy documents and interviews with 
policy actors to address some limitations discussed above; the authors of 
these chapters provide additional information about their methodology. 

Our approach to the study of the architecture of policy knowledge could 
be applied to other cases and domains. We hope scholars will find this 
methodological note useful as they extend this research. 'The database we 
generated through this research project provides an opportunity to explore 
many more questions about policy knowledge. For example, scholars could 
explore the selection process of references into WPs (i.e., why are some 
references, but not others, included in WPs?) Scholars could contribute to 
the mapping of the policy knowledge domain by pointing out the relation- 
ships of co-cited authors (i.e., which authors are co-cited frequently?) Also, 
scholars could examine the titles/abstracts of documents in the database to 
examine topical patterns and frequent labels/words. 
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Note 


1. Following SNA literature we conceptualize documents as actors/nodes. 
However, it is important to note that we do not make any assumption 
about documents having agency or ability to form relationships with others. 
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Policy Borrowing and Evidence 
in Danish Education Policy Preparation: 
The Case of the Public School Reform 
of 2013 


Trine Juul Reder and Christian Ydesen 


Since the beginning of the 1990s, Danish education policy has increasingly 
aligned with transnational trends. Danish student performance in interna- 
tional large-scale assessments (ILSAs) has sparked public and political 
debates concerning the Danish education system (Holm-Larsen, 2010). 
Given the students’ mediocre performance and relatively high public educa- 
tion spending, Danish policymakers have focused on reforming the educa- 
tion system (Imsen et al., 2017), resulting in the initiation of several 
educational reforms in Denmark since 2000. These reforms include the 
2006 public school reform, several reforms addressing primary school cur- 
riculum objectives (2003, 2009 and 2015), the 2007 introduction of 
national assessment tests in primary and lower secondary schools, and finally 
the 2013 public school reform (Danish Evaluation Institute, 2012, 2015). 
This chapter focuses on the 2013! public school reform, which is the 
latest major reform of the Danish public school system and one of the 
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most controversial and hotly debated political reforms in recent years. 
The reform had three overall goals (Danish Government, 2012; Ministry 
of Education, 2014): 


1. The Public School must challenge all pupils to reach their fullest 
potential. 

2. The Public School must reduce the significance of pupils’ social back- 
ground for academic results. 

3. The trust in the Public School and pupil well-being must be enhanced 
by showing respect for professional knowledge and practice. 


These goals were to be accomplished through measures such as longer 
school days, earlier foreign language learning, 45 minutes of exercise 
daily, homework assistance in "homework cafés," and more lessons in 
Danish and math (Ministry of Education, 2014). 

The reform proposal sparked heated public debate concerning the 
reforms content and the scientific evidence behind its elements. Longer 
school hours were a central topic because the change interfered in the 
balance between school and leisure activities. A main argument for longer 
school hours was to improve students’ academic level; however, Danish 
reports on the Progress in International Reading Literacy Study (PIRLS) 
and the Trends in International Mathematics and Science Study (TIMSS) 
results published a mere week after the reform's presentation stated there 
was no scientific evidence indicating a relation between school hours and 
academic achievement (Allerup, 2011; Mejding & Ronberg, 2012). 
Merete Riisager, of the right-wing opposition, consequently called for a 
consultation concerning the knowledge sources underpinning the reform, 
and the debates continued in the media and in Parliament during the 
spring of 2013. 

Another striking point in the reform's presentation was reference to 
ILSAs and foreign school systems’ experiences, particularly the reform of 
Ontario’s school system, which markedly improved its PISA rankings 
(Levin et al., 2008; Coninck-Smith et al., 2015). In 2012, Christine 
Antorini, then Minister of Education, expressed that the upcoming 
reform was largely based on an "inspirational" trip made by Danish 
Parliament representatives to Ontario (Aisinger, 2012) and later repeated 
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that Denmark could learn much from Ontario success (Larsen, 2012; 
Moller, 2012). The reform was introduced following over a decade of 
discussion and negotiations around public school reform, justified mainly 
by the Danish students’ mediocre results in the PISA surveys from 2000 
to 2011 (Cort & Larson, 2015; Egelund, 2008). Regarding ILSAs, only 
596 of Danish pupils were "strong readers," and 1796 lacked both func- 
tional math and science skills (Danish Government, 2012). When the 
Social Democrats won the October 2011 election, the new government 
committed to reforming public school. In December 2012, the govern- 
ment presented a proposal for public school reform, entitled Make a 
Good School Better—Improving the Academic Level of the Public School 
(Danish Government, 2012). 

Given this background, this chapter connects with the theme of this 
volume and contributes to understanding the interplay between interna- 
tional policy trends and national education policy development in gen- 
eral and the role of evidence in the process behind Denmark’s 2013 
public school reform in particular. However, such an analysis calls for 
initial conceptual reflections because discussion about what constitutes 
real evidence often steals the focus (Christensen & Krejsler, 2015). 

As Steiner-Khamsi (Chap. 2, this volume) argues, a reference can be 
understood as “validation of evidence.” Thus, a clear connection is estab- 
lished between references and evidence. Following Steiner-Khamsi, we 
employ a pragmatic approach to evidence, meaning we consider all kinds 
of knowledge sources and information used to inform policy processes 
and create a basis for decision making to be evidence (Cairney, 2015). 
These initial reflections allow for the following overall research questions: 


1. What evidence base underpins the 2013 Danish public school reform? 
2. In what policy context was the evidence base formed and used? 


Methodological Considerations 
and Chapter Structure 


Steiner-Khamsi et al. (2020) emphasized that evidence behind education 
reforms differs vastly. Therefore, we need to remain exploratory and open 
in terms of investigating “which evidence governments actually use when 
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they formulate the policy" (p. 138). We also connect with Ozga's (2019) 
argument about taking the political dimension of policy seriously: 
"Seeing policy as politics focuses on how state policy in its design and 
attempted delivery involves politics, through interests, conflicts, power 
and control—so that politics is an essential element of policy" (p. 21). 

To realize these insights, we employ a mixed-methods design that cre- 
ates an abductive platform between sources of official policy knowledge 
found in a bibliometric analysis of the policy documents and sources of 
unofficial policy knowledge emerging from research, media articles, and 
a contextual reading of policy documents. Additionally, we conducted 
four qualitative interviews with key informants in the spring of 2019 
(Appendix 1) to uncover other sources of knowledge than those explicitly 
cited in the policy documents. 

We start with a bibliometric analysis based on 231 references in the 
preparatory school reform policy documents. This quantitative analysis 
sheds light on the types of policy evidence used in the proposed reform 
and patterns emerging from the reference network of knowledge sources. 
The second part analyzes the policy context framing the formation and 
use of the evidence base. The conclusion summarizes the main findings 
and connects to the overall research questions. 


Source Documents 


For the basis of the bibliometric analysis, we include the reform proposal 
itself (Source Document 1) and the four references of the reform pro- 
posal authored by a government institution: two reports by the Agency 
for the School Council (Source Documents 2 and 3)? and two reports by 
the national sector research agencies The Danish Evaluation Institute 
(EVA) and The Danish National Centre for Social Research (SFI) (Source 
Documents 4 and 5). Unlike the Official Norwegian Reports (Steiner- 
Khamsi et al., 2020), these four reports were not specifically prepared as 
background papers to formulate the school reform policy; however, we 
argue they constitute the best possible data for our bibliometric analysis. 
Table 4.1 displays the ten references of the school reform, where the first 
four constitute the source documents of this analysis. 
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Source Documents 2 and 3 were authored by the Agency for the 
School Council, which was established in 2006 to “follow and assess the 
quality of the primary and lower secondary school and advise the Minister 
of Education" (School Council, 2007, p. 3). It was independent of the 
Minister of Education, and its members were representatives of different 
interests in the elementary school area. Although the council reports were 
produced by a secretariat at the Ministry, its content was decided by the 
council. 

The two sector research reports were authored by SFI (Source 
Document 4) and EVA (Source Document 5), and both institutions were 
partly funded by the government. An independent state institution estab- 
lished under the Ministry of Education in 1999, EVA, is one of the cen- 
tral research institutions working in education policy (Bjerre & Reimer, 
2014). EVA evaluates and researches within the education field at the 
request of other branches (e.g., ministries, local authorities), as well as on 
their own initiative (Danish Evaluation Institute, 2019). SFI was a sector 
research institution under the Ministry for Economic Affairs and the 
Interior. This institution mainly conducted research in welfare state poli- 
cies (e.g., concerning the labor market and family-related issues). In 
January 2017 SFI merged with another public research institution and 
was renamed The Danish Center for Social Science Research (VIVE). 


Qualitative Data 


The qualitative analysis relies on four semi-structured interviews with key 
policymakers behind the reform: a former ministerial official who worked 
on the reform proposal (INF1), a former Parliament member (INF2), 
and two former members of the School Council (INF3 and INF4). The 
informants were directly involved in the reform preparation or in the 
work of the School Council, and the interviews were coded in a data- 
driven approach (Schreier, 2014). The codes are presented in Appendix 1. 
The interviews offer insights into the preparatory work in meetings and 
seminars with stakeholders for which no records or summaries are pub- 
licly available. The informants were anonymized, since the reform sparked 
heavy public debate regarding the evidence behind the reform. Therefore, 
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it was an ethical and strategic choice to give the informants the opportu- 
nity to speak freely without fearing consequences in the form of media 
critique or otherwise. 

To verify the reliability of the interview data, we cross-checked the 
interview information, arguments, and claims through other interviews 
and weighed it against open-source information using source criticism 
procedures found in historical research (Ifversen, 2003). We therefore 
include public statements from key policy officials to connect to the ways 
the reform was presented and justified to the public. We located the 
quotes and articles using Infomedia, a Danish media surveillance com- 
pany that covers all Danish newspapers. 


Mapping the Evidence Base 
of the School Reform 


In this section, we present the main findings of the bibliometric analysis 
in terms of the reference locations, document types, network of refer- 
ences, and publishers. Table 4.2 presents the reference distribution 
between the five Danish source documents and the distribution between 
the document types and location. 

The School Council reports and SFI reports have the most extensive 
citation practices, explained by the nature of these institutions: The 
School Council's (2007) purpose is to “provide documentation for the 
initiatives and actions that contribute to increasing the quality of the 
primary and lower secondary school" (p. 14). Thus, its purpose is to pro- 
vide evidence and disseminate research to the Ministry of Education in 
yearly reports, focusing on specific topics. SFI is generally considered a 
research-heavy institution compared to EVA, which mainly conducts 
empirical research and evaluates specific national initiatives in education. 

We split the references into five document types: reports, books, aca- 
demic, government, and others. The reference distribution reveals that 
the source documents mainly cite government documents (33.7796) or 
reports (29.87%; see Table 4.2). “Government” evidence includes docu- 
ments and reports produced by the government, a ministry, or an agency 
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of a ministry. "Reports" can be produced by research agencies, interna- 
tional organizations (IOs), or private consultancy firms. The "academic" 
literature accounts for 16.0296 of all references, mainly distributed in the 
2012 School Council report (Source Document 3) and the 2011 SFI 
report (source Document 5), both of which concern school leadership. 
Furthermore, the SFI report was commissioned by the School Council 
(Danish National Centre for Social Research, 2011, p. 7). It is therefore 


not surprising that the reports are similar in terms of references. 


Network of References 


Figure 4.1 illustrates the complete network structure and co-citations 
between the five source documents. The references are displayed as 
squares, and Source Documents 1—5 as circles. The size of each note 
reflects its in-degree centrality; the larger the note, the more source docu- 
ments have cited it. The bibliometric analysis shows a relatively low fre- 
quency of co-citations between the source documents: only 16 references 


Mehibye (2010) “The high-performing school = 
How can schools improve the proficiency level 
of students with a weak social background" 


Fig. 4.1 Complete network structure. (Note: Circles represent source documents. 
Squares represent citations. The colors determine whether the citation is regional 
(gray), domestic (white), or international (black). Node size reflects in-degree 
centrality) 
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are shared by a minimum of 2 source documents, and no reference is 
shared by more than 3. 'Ihe report cited by three source documents and 
thus at the center of Fig. 4.1 is Zhe High-Performing School—How Can 
Schools Improve the Proficiency Level of Students with a Weak Social 
Background? (Mehlbye, 2010). It was published by the Danish Institute 
of Government Research (AKF) and commissioned by the School 
Council to examine so-called high-performing schools, focusing on 
schools that improved the academic performance of students with weak 
social backgrounds. The report is thus a classic example of a “what went 
right” approach, where policymakers attempt to find the best practice by 
studying successful examples. Qualitatively analyzing 12 schools, the 
report seeks to deduce what characterizes high-performing schools and 
concludes that, for instance, clear goals, academically competent teach- 
ers, and homework assistance were important for the pupils to succeed 
(Mehlbye, 2010). Since reducing the influence of students’ social back- 
ground on academic results was one of the three goals of the school 
reform, it is not surprising that a report addressing these issues appears 
significant. 

The shared references in the source documents mainly comprise 
research by the government or connected institutions‘ (11 of 16 refer- 
ences) or international evidence from either IOs or international edu- 
businesses (3 of 16 references; see Appendix 2 for a list of all 16 
co-citations). The low frequency of co-citations in Source Documents 
2—5 reveals that the 4 reports behind the school proposal represent spe- 
cialized knowledge: they each focus on specific subjects used to substanti- 
ate the different reform elements. The 2011 School Council report 
focuses on transfer between primary and secondary education, schools’ 
collaboration with municipalities, and after-school classes, whereas the 
2012 School Council report is concerned with school management. The 
EVA report (commissioned by the School Council) concentrates on dif- 
ferentiated teaching, evaluation, and teacher professionalism, and the SFI 
report (commissioned by the Ministry of Education) centers on school 
leadership. This picture of specialized knowledge is similar to Norway's 
case, in which the Norwegian Official Commissions (NOUs) generate 
“highly specialized knowledge” that allows the government to “selectively 
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transfer what was produced at the commission level to the political level" 
(Steiner-Khamsi et al., 2020, p. 128). 


International Inspirations: Academic Research 
and Large-Scale Assessments 


A remarkable pattern appears when we look at the international and 
domestic references individually, as in Table 4.3. Namely, the academic 
references consist almost exclusively of international research. This find- 
ing is in line with a case study of Norwegian policy advisory commissions 
by Christensen and Holst (2017), who found growing reliance on what 
can be categorized as academic knowledge in general and (in the Norwegian 
context) on international academic knowledge in particular. Although 
Denmark has a long tradition of educational research going back to 
World War I (Gjerloff & Jacobsen, 2014), this research surprisingly does 
not seem to appear in the preparation of the 2013 school reform. The 
international academic references are mainly centered on the topic of 
educational leadership, and the majority are referenced in Source 
Documents 3 and 5, both reports concentrating on school leadership at 
different levels (school, municipal, and national). The most cited interna- 
tional journals are Educational Administration Quarterly and. Journal of 
Public Administration Research and Theory, both of which focus on educa- 
tional leadership and management, as well as public administration 
(Oxford Academic, 2020; Sage, 2020). 

The remaining international references are distributed mainly as 
reports (21.4396) and books (25%). Of all international reports (z = 23), 
the majority come from IOs (14) and consultancy firms (4)? The OECD 


Table 4.3 International, regional, and domestic references distributed by types of 
knowledge 


Journal 
Report Book article Govt Others 
Domestic references 34.29% 9.29% 0.71% 49.29% 6.43% 
(n = 140) 
Regional references (n 2 8) 37.50% 25.00% 0.00% 37.50% 0.00% 
International references 21.43% 25.00% 42.86% 8.33% 2.38% 


(n = 84) 
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(2010b, 2010c, 2010d, 2011a, 2011c) is the most cited IO, and the cita- 
tions mainly refer to PISA. This emphasizes the importance of IO evi- 
dence, especially concerning PISA, in Danish school policymaking in 
preparing and legitimizing the reform. We detail this in the subsequent 
qualitative analysis, focusing on the use of international experience and 
lessons in the 2013 school reform. 


Reliance on Government-Commissioned Evidence 


While the international citations mainly refer to academic and IO evi- 
dence, domestic citations largely refer to government and public institu- 
tion reports. Figure 4.2 shows the 11 most cited publishers, 6 of which 
are the following public institutions: Ministry of Education, EVA, SFI, 
the Danish Government, the School Council, and AKE” The Ministry of 
Education is by far the most cited. 

As seen, the field of social policy (including education) in Denmark 
contains a range of institutions aimed at providing evidence for policy- 
makers. The bibliometric analysis reveals three of these institutions (EVA, 
SFI, and AKF) are among the most commonly cited in the preparations 
for the public school reform. One explanation for this finding is that it is 
common practice for institutions to cite themselves where possible. SFI 
and EVA are the authors of Source Documents 4 and 5, respectively, and 
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Fig. 4.2 Most cited publishers 
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the EVA evaluations on compulsory education were all assigned by the 
School Council, author of Source Documents 2 and 3 (OECD, 2011b; 
School Council, 2007). 

Research from public agencies thus constitutes a large part of the refer- 
ences.? The evidence produced by these institutes is mostly commissioned 
by public institutions, such as ministries and municipalities. Bjerre and 


Reimer (2014) explained, 


the research institutes have established their own system of peer review and 
quality check [...] and thus the reports move directly from the research 
agencies into the political, administrative, and public process, without the 
intermediation of a scientific community of critical readers and the general 
peer review system [...]. (p. 85) 


The majority of the evidence produced by these institutions can be cate- 
gorized as strategic evidence, which is evidence commissioned by the gov- 
ernment (or a government institution) or carried out by a public 
institution. However, there are two reservations. First, not all research by 
public agencies is strategic; for example, institutions such as EVA and SFI 
also conduct research for external parties, such as private funds (e.g., 
Danish Evaluation Institute, 2020). These agencies also conduct what 
they call independent? research, which is research funded by their annual 
budget. Second, strategic evidence is sometimes produced by private 
organizations, companies, or universities. To establish the extent of stra- 
tegic evidence in the 2013 public school reform, we therefore take a closer 
look at the references. 

To identify how strongly the school proposal relies on strategic evi- 
dence, we thoroughly examine the references. First, we exclude all refer- 
ences to academic articles. We then select all references that could be 
categorized as strategic evidence based on the publisher. Finally, we exam- 
ine each reference individually to establish the report's funding and com- 
missioning (see Appendix 3 for selection criteria). 

Our results show that 63 of the 231 references count as strategic evi- 
dence produced in Denmark.'^ The reports were mainly commissioned 
by the Ministry of Education or the School Council and carried out by 
various actors and organizations, mostly EVA, the Agency for the School 
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Council, or SFI. A picture can be drawn of the remarkable evidence com- 
position where strategic and politically influenced evidence accounts for 
as much as 27.2796. 


Summary: Types of Policy Evidence 


According to the reference analysis, we found that the Danish public 
school reform relies on at least three types of evidence, which we catego- 
rize as academic, strategic, and IO evidence. The first two categories are 
inspired by Bjerre and Reimer (2014), who distinguish between strategic 
and academic evidence in their analysis of Danish teacher training 
programs. 

Our analysis of the references in the five source documents shows that 
strategic evidence constitutes a large part of the references. Research 
agencies such as EVA and SFI are behind many of them. Academic evi- 
dence constitutes a significant percentage (16.0296) of the references, 
almost all of which are published in international journals. The interna- 
tional academic references focus mainly on leadership, a central part of 
the reform. The IO evidence has been a large source of inspiration and 
legitimization of the school reform. The analysis shows the OECD is the 
fourth most cited publisher. Furthermore, Aarhus University, which pro- 
duces the Danish TIMSS, PIRLS, and OECD reports on education, is 
also among the most frequently cited publishers. The reform proposal 
references OECD data to justify the need for reform: Danish students do 
not have proficient reading skills, “have problems in mathematics,” and 
are not "good enough" in science (Danish Government, 2012, p. 7). 

The bibliometric analysis reveals reference patterns that we explore fur- 
ther in the following two qualitative sections. The first research question 
(RQ1) relates to the knowledge sources underpinning the reform. We 
have shown that international academic evidence, IO evidence, and stra- 
tegic evidence constitute a large part of the references. However, although 
the analysis enables us to identify reference patterns, it does not tell us 
much about the context or reasons for the patterns. Moreover, it reveals 
only evidence explicitly referenced in the policy papers. As shown, in 
Denmark case, the number of explicit references is significantly lower 
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than for other Nordic countries, which raises the question of whether the 
2013 Danish public school reform relied on other types of evidence than 
what appears in the policy paper references. 

Our analysis also shows that international references constitute a large 
part of the references (36.3696) and that these are partly academic and 
partly stem from IOs, particularly the OECD. In the second qualitative 
section, we analyze the policy context in which the evidence base was 


formed and used (RQ2). 


Evidence Beyond the References 


Scholars have argued that commissions play an important role in policy 
preparation in Scandinavia today, especially in Norway and Sweden 
(Christensen & Holst, 2017; Holst & Molander, 2018; Steiner-Khamsi 
et al., 2020). In contrast, such a practice is seldom seen in Denmark 
(Christensen et al., 2009). With the 2013 public school reform, no com- 
mission was established to prepare the bill, and there were no reports or 
documentation of the evidence, except for the references in the proposal 
itself (INF1, INF2, INF4). 

It is worth noting that the new government's memorandum of under- 
standing between the three participating parties expressed the need for 
public school reform some two years before the reform (Danish 
Government, 2011). This indicates that, even before the government 
won the election, the reforms main contents had already been negotiated 
between the Social Democratic Party, the Socialist Peoples Party, The 
Red-Green Alliance, and the Danish Social-Liberal Party. A former 
Parliament member explained that preparing the reform proposal "has a 
long history before the government was established. In the years leading 
up to the election in 2011 [...] there was close coordination between the 
spokespeople of the opposition parties" (INF2) and discussions about the 
reforms content, particularly the idea of a comprehensive school, inclu- 
sive of all children, regardless of wealth, social background, or abilities 
(Imsen et al., 2017). If anything, these preparatory tracks indicate the 
ideological priorities of the new government. These priorities are not 
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rooted in evidence as such but testify to the politics of policymaking. 
One informant went so far as to say, 


In my opinion, an important point to make is that the Danish school 
reform is not characterized so much by research evidence. Its main ele- 
ments are based on political and ideological thinking rather than a scien- 
tific foundation. And that is probably one of the reasons there are so few 
references. In addition, and especially in the pedagogical area, there has not 
been a strong tradition of assembling a common platform of research evi- 
dence. [...]. So that is also part of the explanation. It is different in Norway 
and Sweden. (INF4) 


The quantitative analysis of references is based on official policy evi- 
dence and an assumption that evidence constitutes the core building 
blocks of policy development, that is, that policy development is a fairly 
rational process. Instead, we argue that, in Denmark’s case, we need to 
adopt a more nuanced view of what constitutes policy evidence and to 
reflect on the role of ideology (i.e., politics) in the processes surrounding 
all applications of evidence in the preparation of the 2013 reform. To 
start, we argue that, in addition to academic, strategic, and IO evidence, 
two other types of evidence are fundamental in the preparations of the 
public school reform: stakeholder evidence and practice-based evidence. 


Stakeholder Evidence 


After the 2011 election, the school reform proposal work took three dif- 
ferent paths. The first was a partnership with the stakeholders of public 
school, who met in three seminars between October 2011 and December 
2012 to discuss the main topics of the reform: 


The public school reform was described quite in detail in the government 
program, but it had to be translated into an actual bill. And [...] a partner- 
ship with the main stakeholders was established. [...] There were three 
seminars which were theme-based on the main topics [...]—it was like 
inspiration for us to see if we could create—well, we didnt call it a consen- 
sus conference, but it was building on the idea: If everybody was there and 
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we were talking about the content [of the reform]—if we could say, “there 
are these five points that we agree on.” (INF2) 


The seminars were organized by the Ministry of Education, and the six 
stakeholders were the teachers’ union, the school principals’ union, the 
union for early childhood teachers and youth education, the interest 
organization of the municipalities called Local Government Denmark, 
the students’ union, and the parents’ organization (INF2). Second, there 
was a working group across several ministries, including the Ministry of 
Education, the Ministry of Higher Education and Science, the Ministry 
of Economy and the Interior, and the Ministry of Finance. Third, an 
office dedicated to the reform work was established in the Ministry of 
Education (INF1). 

According to an informant from the Ministry of Education, the cul- 
ture of negotiation is typical for Danish education policymaking: 


Its funny with Denmark and Sweden—we [the Danish people] have a 
culture of being merchants and traders and adjusting things along the way, 
and we have flexibility and close collaborations, whereas Sweden is like a 
big industrial nation—they produce cars—they take the commission work 
and then they lead it into the government and decide something. ... So, it's 
like an industrial nation way of thinking, which they also adopt in their 
policy development. And in that case, we are more like merchants ... we 
negotiate something that everybody has a share in. And during the process, 
we collect some knowledge and try to establish a broad ownership. (INF1) 


On a more objective note, a recurring observation in the Danish his- 
tory of education is that reforms of the public school system have involved 
broad cross-party compromises to secure political continuity and broad 
support from stakeholders to secure involvement (Coninck-Smith et al., 
2015; Gjerloff & Jacobsen, 2014). The observations in this section are 
indicative of the political culture surrounding the reform process. 
Stakeholder evidence is found to play a role, even if the framing of stake- 
holder evidence can be considered an expression of political and ideologi- 
cal priorities. These observations gain currency when we consider the role 
of practice-based evidence. 
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Practice-Based Evidence: Bottom-Up 
and Experimental Projects 


In August 2012, the Minister of Education Antorini launched the project 
called New Nordic School, inviting institutions from the educational sec- 
tor (from pre-primary education to upper secondary education) to par- 
ticipate. Ihe New Nordic School project was part of the ideological basis 
of the school reform and constitutes another type of evidence ground- 
work for the reform, which we denote practice-based evidence. The proj- 
ect’s three overall objectives—identical to those of the school reform—had 
been determined beforehand. In an annual meeting held in the town of 
Soro, practitioners and experts discussed the challenges for public school, 
focusing on different themes, and determined ten objectives. The 2012 
Soro meeting centered around the New Nordic School project later, a 
manifest was released with these ten points, as well as three overall objec- 
tives (Ministry of Education, 2012). By November 2012, over 350 insti- 
tutions had applied to participate in the project (Olsen, 2012). The 
institutions committed themselves to initiate change processes inspired 
by the project's manifest and objectives. Across the country, “networking 
days” were held where institutions discussed and developed their indi- 
vidual projects. 

Another example of practice-based evidence involves the tradition for 
experimental project work in Danish public schools (Andreasen & 
Ydesen, 2015; Coninck-Smith et al., 2015; Gjerloff & Jacobsen, 2014). 
Since the 1920s, Danish school policy development has been character- 
ized by experimental projects in schools and teacher training colleges, a 
practice later known as the Danish Model (Coninck-Smith et al., 2015, 
pp. 112-115). In the case of the 2013 public school reform, an infor- 
mant explained that “in the content of the school reform, there wasnt a 
single new thing which hadn't been experimented in schools" (INF2). 
Examples of experimental projects with elements of the reform include a 
2005 project about homework cafés in 15 Copenhagen schools (Vogt- 
Nielsen & Hansen, 2005) and a project about extended school hours 
(heldagsskole), tested in 12 schools across the country and evaluated in 
October 2012 (Rambøll, 2012). 
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As seen, increasing schooling hours was one of the most radical and 
problematized elements of the reform. Interestingly, extending the school 
day is not referred to as heldagsskole, and Ramboll’s (2012) evaluation is 
not mentioned in either of the source documents. The reason could be 
that the report concludes there is no significant evidence supporting the 
statement that extended school hours improve academic performance. 
Furthermore, a former member of the School Council was skeptical 
about Ramboll’s evaluation (INF4), suggesting the reform was based on 
the idea that 


[...] if you wanted to help students with a weak social background, then 
the school would have to—to a larger extent—deal with all learning activi- 
ties. And you had to leave fewer things up to the parents—less home- 
work .... [...] This is not based on any scientific evidence—this was 
definitely driven by ideology and visions [...]. (INF4) 


So far, our analysis has shown that practice-based evidence is an impor- 
tant part of the evidence base of the school reform. Other research has 
demonstrated that the role and involvement of practice-based evidence in 
education reforms has a long history in Denmark. However, our analysis 
has also revealed a strong tendency toward cherry-picking. One infor- 
mant said, “The politicians are very interested in evidence—if it matches 
their opinions” (INF4). This point takes us back to the ideological and 
political elements in the reform process. 


Using International Evidence 
in the Preparation of the School Reform 


More than in the other Nordic countries, Denmark’s public school reform 
draws on international evidence sources, namely international academic 
articles and IO reports. The bibliometric analysis concludes that, in spite 
of a long tradition of Danish educational research, the source documents 
cite mainly international academic research. This could be because 
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Danish researchers often publish in international journals, in which case 
the data are still considered international research. While this observation 
may hint at an error in the analysis, the qualitative interviews indicated 
there could be another reason for the "missing" Danish education 
research: 


I still think there is something missing [in Danish education research]. 
There are a lot of things that have not been properly examined. ... I thought 
it was a big challenge that we didn't have any study saying, “Ok, what will 
the effect be if we do A or B in some area?" ... And some things are just not 
properly scientifically examined and others are but [...] in studies where it 
is difficult to conclude anything about causality and effect. (INF3) 


Another informant expressed that there was just not enough Danish edu- 
cation research (INF1), or at least not the kind requested by 
decision-makers. 

The evidence provided by IOs has impacted the preparation of the 
public school reform both directly and indirectly. Since the first ILSA in 
which Denmark took part was published, international experience has 
played an important role in Danish education policymaking (Andreasen, 
2019). First, as Addey and Sellar (2018) noted, ILSAs often serve to legit- 
imize the need for reform, as was also the case of the public school reform 
(INF2), where PISA was used to “diagnose” the problems (INF1). The 
public school reform came after more than a decade of debates about the 
state of the public school system and even an earlier attempt to propose 
school reform by the former Danish Government (2010). The ground 
was thus laid for the reform, a point that lends support to what Dobbin 
et al. (2007) refer to as the constructivist mechanism of policy diffusion, 
emphasizing the importance of public policies becoming socially accepted 
for their diffusion ability. However, the ILSAs themselves usually do not 
indicate any specific solution: 


As soon as you need to find solutions, then you have to render it probable 
that you causally will get the effect that you claim that you will get. And 
how can you get that? You need to have some studies where you have 
looked at some changes, done some intervention, and documented that 
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this intervention had the planned effect. That is causality. And [those stud- 
ies] are typically carried out in a different context [than the Danish one]. 
Then you look at Ontario—they did this reform and had that result. And 
then we add this idea about causality—but you have to consider whether 
you can have the same effect in a Danish context. And you have to be 
extremely careful. (INF1) 


Nevertheless, the OECD's PISA program set the agenda for which 
countries to look to when preparing school reforms. 'Ihis holds true not 
only for the 2013 public reform, but also for the school policy reforms 
and development in the years leading up to the reform: 


Under the Anders Fogh" government, there was a lot of focus on 
Singapore—that was where everybody was going, and you had to learn 
from the miracle of Singapore. Then the wind was blowing towards west, 
and it was Ontario that we all were looking towards. (INF2) 


Thus, although Singapore functioned as a positive reference society dur- 
ing the right-wing government, it was replaced by Ontario after the elec- 
tion, with some criticism of the past focus on Asia. As explained by a 
former Parliament member, 


The Asian school system is very far from the Danish one, which makes it 
really difficult to translate. With the Canadian model ... it was easier to be 
inspired by the way they have built their school reform—and transfer it 
into the Danish context. (INF2) 


Policy Borrowing: Inspiration from Ontario 


The inspiration from the reform of Ontario school system in 2003 is 
reflected in media reports, as well as in the interviews (Fuglsang, 2012; 
Moller, 2012; Sendergaard, 2012). In September 2011, the School 
Council (2012) visited Ontario to study the reasons behind its successful 
school reform in 2003. One year later, members of the Danish Parliament 
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Committee for Children and Education (consisting of the spokespersons 
for education from each political party) went on a similar field trip to 
Ontario to meet with politicians, researchers, and teachers (Ministry of 
Foreign Affairs, 2012). Furthermore, Mary Jean Gallagher, Chief Student 
Achievement Officer in Ontario, was invited to the Danish Ministry of 
Education during the reform's preparation, as well as to a 2012 New 
Nordic School meeting (INF2; INF4). Because the representatives of all 
political parties had visited Ontario, it became a common reference point 
for the negotiations, with politicians referring to examples from Ontario 
when arguing (INF2). Not only did Parliament members and the School 
Council go to Ontario, but individual schools and municipalities inter- 
ested in the “miracle of Ontario" also went on study trips to discover the 
factors for success (Fuglsang, 2012; Jacobsen, 2012). Through this 
engagement with stakeholders, municipalities, and the public, general 
social acceptance of the need for school reform was established. 

The 2013 public school reform is a classic example of policy borrowing, 
in which reference (to Ontario) and transfer (of educational policies) 
occur together, lending authority to the reform as a package of best prac- 
tice policies (Phillips & Ochs, 2004; Waldow, 2017). The three goals of 
the Danish public school reform were nearly identical to the goals of the 
2003 education strategy of Ontario: “raising the bar for all students, 
reducing achievement gaps and restoring public confidence in the 
publicly-funded school system” (Ontario Ministry of Education, 2010). 
However, while policy borrowing, in the terminology of Phillips and 
Ochs (2004, p. 778), begins with the stage of “cross-national attraction,” 
how such “attraction” comes into play must be addressed. As Simons and 
Voß (2018) have argued, policy solutions do not always follow policy 
problems. The concept of instrument constituencies accounts for the fact 
that "instrumental options and their consideration as viable solutions to 
certain problems does not necessarily [...] follow the diagnosis of prob- 
lems—neither chronologically, nor in terms of stages in a sequenced pro- 
cess of rational analysis and problem-solving” (p. 15). While the 
qualitative interviews pointed to a common understanding of the policy 
solution (the school reform) following a problem (Danish students’ poor 
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performance) (INF1; INF2), the role of IOs, especially the OECD, has 
clearly had a large impact on both defining the problems and providing 
possible solutions. Hence, it was the performance of students in PISA 
that sparked the need for education reform in both Ontario and Denmark. 


Concluding Discussion 


In this chapter, we have explored the landscape of evidence behind the 
2013 Danish school reform by combining bibliometric and qualitative 
analysis. This particular reform has been the subject of intense debates 
concerning whether it was based on any evidence or whether it was a 
purely political project, with no documentation of the effect of the major 
changes it induced in Danish public school. Asking what evidence under- 
pinned the reform could be interpreted as support for one side of this 
debate. However, as we argued, we have chosen to employ a rather broad 
and inclusive definition of the concept of evidence. We therefore reviewed 
all types of knowledge sources as part of a landscape of evidence in the 
reform process. 

Combining the bibliometric and qualitative analysis has enabled us to 
unravel multiple dimensions of the preparatory work behind the 2013 
Danish school reform. The bibliometric analysis allowed us to examine 
the reference patterns of the official policy knowledge behind the school 
reform. A key finding pertains to the academic evidence, which played a 
significant role in the bibliometric composition of evidence. Our analysis 
clearly reveals a particular clinical type of academic evidence considered 
by decision-makers to be useful to education reform. The prevalent type 
of academic evidence can be characterized as evidence-based and “what- 
works." This evidence provides hard-core data and/or is concerned with 
revealing best practices. Moreover, we saw how academic evidence does 
not stem from Danish education research in general, but from a very 
particular string of research. Theoretically, this valorization of a particular 
kind of academic evidence could be considered an expression of competi- 
tion in policy borrowing (Dobbin et al., 2007). The 2013 school reform 
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seems to revolve around an idea of international competition, living up 
to international standards, and importing best practices to quench a fear 
of falling behind (Krejsler, 2019). From a broader perspective, the thesis 
about the competitive state could help further understand the implica- 
tions. According to Pedersen (2010), the competitive state mostly views 
education as an investment in the states competitive performance on the 
global level. 

Furthermore, the analysis reveals that the reform largely references 
international sources and strategic evidence. Categories such as domestic 
academic literature or references to documents and data from Nordic 
countries are almost non-existent in the policy evidence explicitly refer- 
enced in the policy papers behind the school reform. ‘The strategic evi- 
dence is provided by public research institutes and ministry units (e.g., 
EVA, SFI, AKF the School Council) and private actors and universities 
(e.g., Rambell and Aarhus University). 

The number of explicit references in the policy papers behind the 
reform is significantly lower than in the other Nordic countries. However, 
as our analysis shows, this does not necessarily mean that the reform did 
not rely on any type of evidence. In the qualitative analysis, we con- 
structed additional analytical categories to account for two types of policy 
evidence (i.e., stakeholder evidence and practice-based evidence) not 
apparent in the bibliometric analysis, but important to the reforms prep- 
aration. Ihe New Nordic School project allowed more than 350 institu- 
tions (e.g., schools, kindergartens) to work individually with the reform's 
goals. Meanwhile, meetings with the key public school stakeholders took 
place at the Ministry of Education, where the reform was discussed and 
developed. However, the reform elements were also largely based on ide- 
ology. Work on the reform began even before the government took office 
in 2011, and the key elements were determined before the official work 
began that same year. This suggests that the official portrait of evidence 
behind the reform has a distinct cherry-picking flavor, what Pawson 
(2006, p. 7) called “policy-based evidence" to describe research that “trav- 
els straight from ideology to policy recommendations via the cherry- 
picking of evidence." 

The case of the 2013 Danish school reform is also a clear example of 
policy borrowing (Phillips & Ochs, 2004), and we saw how a 
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constructivist mechanism of policy diffusion supports the strong inspira- 
tion and lessons drawn from Ontario. In defining the constructivist 
mechanism, Dobbin et al. (2007) contends how "policy makers play fol- 
low the leader by mimicking the countries that appear to be doing best” 
(p. 452). It is remarkable how politicians, civil servants (officials), and 
researchers associated with the general reform process and the specific 
bodies conducting the reforms preparatory work were all swayed by the 
results and methods of the Ontario education system. Theoretically, the 
reform was underpinned by a powerful narrative constructed by key 
agents about the wonders possible in education following the Ontario 
recipe. But the reform process also highlights the OECDS influence in 
both legitimizing the need for reform (through PISA) and indirectly 
pointing to the solution in terms of Ontario as a positive reference society. 

Thus, the evidence landscape emerging from this analysis is very 
uneven. It consists of distinct plateaus of evidence-based and what-works 
types of evidence underpinned by significant policy diffusion from 
Ontario, OECD framings of what counts in education, strategic evi- 
dence, and a good portion of political ideology. Ihe absence of broader— 
and often critical —Danish education research constitutes the valleys of 
the landscape. While stakeholder evidence and practice-based evidence 
were allowed voices in the reform process, the precise impact and signifi- 
cance of this type of evidence remain obscure. 
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Appendix 1: Coding of Qualitative Interviews 


Table 4.4 Coding of qualitative interviews 


1 General education policymaking in 1a 
Denmark and public school reform 1b 

1c 

1d 


2 International inspiration: Inspiration from 2a 
the global education space in education 


policymaking in Denmark 


2b 


2c 
2d 


3 The reform process 3a 


3b 


3c 
3d 


Table 4.5 Informants of qualitative interviews 


INF1 


INF2 
INF3 
INF4 


Missing academic research 

How Danish education 
policymaking works 

School Council 

Absence of references 

Use of international 
references in the public 
school reform 

Use of data from IOs and 
international consultancy 
firms 

Inspiration from Ontario 

Nordic inspiration 

Evidence behind the reform 
(generally) 

Negotiations about the 
reform—before and after it 
was presented 

Reform proposal of 2010 

Nordic School, bottom-up, 
and experimental projects 


Former Ministry of Education employee, involved in the school reform 


preparation 


Former Parliament member, involved in negotiations about the reform 


Former member of the School Council 
Former member of the School Council 


Note: Conducted May-June 2019 
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Appendix 3: Methods for Counting 
Strategic Evidence 


Step 1 


First, we selected all references identified as reports, books, government- 
issued decrees or guidelines, or Other. References categorized as academic 
articles were excluded since they cannot be strategic. 


Step 2 


Of these, we identified and isolated the references from publishers likely 
to have produced research commissioned by the government or a govern- 
ment institution. 

Public research institutions, councils, or ministries 


* EVA 

* SEI 

* AKF 

* Municipal and Regional Evaluation Institute (KREVI) 

* Danish Centre for Teaching Environment (DCUM) 

° Skolerádet [School Council] or Agency for Skoleradets formandsskab [the 
School Council] 

* Ministry of Education (Undervisningsministeriet) 

° Ministry of Health (Ministeriet for Sundhed og forebyggelse) 

* Ministry for Children and Social Affairs (Borne- og socialministeriet) 

* Danish Government (Regeringen) 

° Ministry of Higher Education and Science (Uddannelses- og 
forskningsministeriet) 

* Ministry for Taxation (Skatteministeriet) 


Private research institutions 


° Rambøll 
* TNS Gallup 


* Radius Kommunikation 
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Universities and higher education institutions 


e Aarhus University 

* Copenhagen Business School 

* University of Southern Denmark 
* University College Nordjylland 


Unions and interest organizations 


* Lederne 


Publishing houses 


* Dafolo 

* Academia 

* Fremad 

* Jurist- og Okonomforbundets Forlag 
* Odense Universitetsforlag 


Other 


* Aarhus University Hospital 
° Folkeskolen.dk 


We thus have 138 references. 


Step 3 


Then we checked each of the 138 references individually to establish 
whether that particular report or book would be categorized as strategic 
evidence, for example, as follows: 


1. All reports commissioned by the School Council are counted as stra- 
tegic evidence. These are reports from EVA, SFI, and AKF 
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2. Three reports from Rambell were counted as strategic evidence. Two 
were commissioned by the Ministry of Education, and the third was 
commissioned by the Ministry of Finance. 

3. Thirteen SFI reports were counted as strategic evidence, commis- 
sioned by a ministry, a commission, the School Council, or SFI itself. 
Since the independent research SFI conducts is provided for by the 
annual Finance Acts, the knowledge production is counted as strategy. 

4, One SFI report was not counted as strategic because it was commis- 
sioned by Bikubenfonden, an independent, commercially operating 
foundation. 


Result 


Among the 231 references, we identified 63 strategic evidence references. 
References cited by multiple sources were counted only once. 


Notes 


1. The reform was passed by Parliament in June 2013, becoming effective 
August 2014. The reform is therefore sometimes called the public school 
reform of 2014 (Danish Government, 2013). 

2. The goals were translated into English publication by the Danish 
Ministry of Education (2014), explaining the reform’s content and 
objectives. The quotes and text passages in Danish were translated by 
Trine Juul Reder and Christian Ydesen, unless otherwise stated. 

3. The Council for Evaluation and Quality Development of Primary and 
Lower Secondary Education (Rådet for Evaluering og Kvalitetsudvikling 
af Folkeskolen) was directed by the Agency for the Council for Evaluation 
and Quality Development of Primary and Lower Secondary Education 
(Formandsskabet for Rådet for Evaluering og Kvalitetsudvikling af 
Folkeskolen), which we refer to, respectively, as the School Council and 
the Agency for the School Council. 

4. These are for instance the School Council, EVA, SFI, and AKE 

5. More specifically, out of the 23 reports, 14 are from IOs, 4 from private 
international companies, 2 from non-governmental organizations, 2 
from foreign governments, and 1 from a university. 
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. Additionally, one citation refers to Education at a Glance 2010 (OECD, 


20102), one to an OECD review on evaluation and assessment from 
2009 (OECD, 2009b), one to the OECD Teaching and Learning 
International Survey 2009 (OECD, 20093), one to the report Preparing 
Teachers and Developing School Leaders for the twenty-first Century — 
Lessons From Around the World (Schleicher, 2012), and one to a 2011 
OECD review on evaluation and assessment in Denmark 
(OECD, 2011b). 


. We distinguished between ministries and the government when entering 


the publisher of each document. However, both are coded as government- 
issued decree, guideline, or report. 


. Aside from the three abovementioned institutes, the Danish Evaluation 


Institute for Local Government (Det kommunale og regionale evaluering- 
sinstitut [KREVI]) and the Danish Centre for Teaching Environment 
(Dansk Center for Undervisningsmiljo [DCUM)]) also appear among the 


references. 


. However, since the public research institutes are funded by the annual 


finance acts, the knowledge production can be considered strategic. 
Furthermore, the Agency for the School Council had the authority to com- 
mission official evaluations carried out by EVA (School Council, 2007). 


. When accounting for strategic evidence, we exclude IO reports. Although 


these can be considered strategic (i.e., they are sometimes commissioned 
by national governments), in our analysis, they count as a separate form 
of evidence, JO evidence, due to its particular significance in the context 
of Danish school policy. 


. The Danish expression enhedsskolen, or “comprehensive school,” refers to 


a school for all children, regardless of social background, wealth, or abili- 
ties. According to Imsen et al. (2017), this school model is typical for the 
Nordic countries, which implemented this model between the 1950s 
and 1970s, with the values of "social justice, equity, equal opportunities, 
inclusion, nation building, and democratic participation for all students, 
regardless of social and cultural background and abilities” (p. 568). 


. Anders Fogh Rasmussen was the Danish prime minister from 2001 to 


2009, succeeded by his colleague (of the same political party) Lars Lokke 
Rasmussen (2009—2011). 


. ICCS is an acronym for International Civic and Citizenship 


Education Study. 


. The Leaders (Lederne) is a Danish union. The organization does not have 


an official English name, but the direct translation means "leaders." 
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Since the 1990s, national policy reforms have been increasingly more 
influenced by transnational actors and the global context (Rizvi & 
Lingard, 2010). International organizations have become particularly 
active players in setting policy direction on a national level (Morgan & 
Volante, 2016). For example, in the field of education, organizations 
such as the World Bank, the United Nations Educational, Scientific and 
Cultural Organization (UNESCO), and the Organisation for Economic 
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Co-operation and Development (OECD) have been prominent in engag- 
ing in national education policy making (Mundy et al., 2016). Their 
instruments of assessment enable them to compare outcomes across 
countries, identify successful practices, and define quality standards that 
can serve as references for subsequent policy adjustments. In this way 
they contribute to setting the rules of the game through "governing by 
comparison" (Martens & Niemann, 2013, p. 317). At the same time, 
expert advice and evidence is widely sought by national policy actors as 
support for framing and legitimizing complex decisions. This has led to 
critical voices claiming for instance that the unelected actors have gained 
power that challenges the conventional decision-making processes (Viber, 
2007) and that high-level strategic thinking may be outsourced from the 
national to the supranational level (Lawn et al., 2011). 

Since the first Program for International Student Assessment (PISA) 
results were published in 2000, many have considered Finland a model of 
educational success. Indeed, the nation is widely used as a reference soci- 
ety (Bendix, 1978, as cited in Waldow, 2017; Waldow & Steiner-Khamsi, 
2019) for standards of good practice by countries around the world, 
including other Nordic countries such as Norway (see, e.g., Sivesind, 
2019), that are looking to improve their education performance. 
Although Finland has had a strong tradition of adopting state-led policies 
in education, the extent to which international organizations exercise 
their influence on the national level is a subject of debate in the Finnish 
context as well. 

In this chapter we address the recurring debate in comparative educa- 
tion on the extent to which education is shaped by national or interna- 
tional influences and explore it from the point of view of expertise and 
evidence. Our focus is on determining the kind of evidence Finland draws 
on when endeavoring to improve its schools and learning outcomes and iden- 
tifying whose expertise is valued most as evidence in this process. As an exam- 
ple of such a reform, we examine the policy-making process that produced 
the 2014 National Core Curriculum of Finland. We discuss our findings 
in light of previous research on education policy making in Finland to 
elucidate the two opposing strands of the debate—one stressing the state- 
centeredness of education policy making in Finland and the other claim- 
ing that transnational organizations and international expertise and 
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evidence are gradually gaining more leverage in the field. To overcome 
the dichotomy between the two opposing forces— national and interna- 
tional—we view global as something that is constructed within the local 
(Massey, 2005; Sassen, 2007, 2013). We use Larsen and Beech's (2014) 
suggestion to focus on "networks, connections and flows" (p. 75) in 
researching educational transfer and Eyal’s (2019) idea of expertise as 
located in relationships and networks between individuals. Our interpre- 
tative framework and our choice of data and methods of analysis stem 
from the definition of evidence by Paul Cairney (2016, p. 3): "Evidence 
is an argument or assertion backed by information." Based on these theo- 
retical starting points, we examine the bibliometric references as informa- 
tion that supports the arguments in the most prominent policy documents 
in the reform process. 

The chapter begins with a discussion on the meaning and role of evi- 
dence and expertise in current policy development, with a focus on the 
context of education policy and politics. We then elaborate on the con- 
text of Finnish education policy making and the argumentation in the 
relevant literature. Next, we present the research design and results. 
Finally, we conclude the chapter by discussing the findings of our research 
in relation to our interpretative framework and presenting our 
conclusions. 


Evidence and Expertise in Education 
Policy Making 


As the world has become increasingly interconnected and the issues fac- 
ing policymakers more complex and global, government authorities 
worldwide have grown to rely extensively on expert advice to inform the 
decisions they must make. Seeking outside opinions has become second 
nature to policy making in modern democracies that strive to perform 
well (Holst & Molander, 2019; Maasen & Weingart, 2009; Moore, 
2017). At the same time, the pace and scope of information flow is con- 
stantly escalating. It has become increasingly challenging, for instance, to 
maintain a working knowledge of the vast and increasing amount of 
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scientific output. Consequently, one key task of the experts and expert 
organizations today is to select the knowledge relevant to address the data 
and information needs of politicians and policymakers (Stanziola, 2012; 
Wolscheid et al., 2019). 

The role of knowledge in policy making and politics has become more 
visible due to the rise of the evidence agenda (Wolscheid et al., 2019, 
p. 273). Politicians and policymakers are expected to base their decisions 
not on intuition and beliefs but on objective and reliable information— 
in other words, actual evidence. The role of experts in providing this 
knowledge has become central. Contrary to popular belief about what 
constitutes expertise, Eyal (2019) argued that expertise is not a set of 
skills or knowledge an individual or a group possesses; instead, expertise 
depends on outside recognition, which qualifies what experts do as 
“expertise.” To be perceived as an expert, one needs to master "the disci- 
plinary knowledge system composed of abstractions and general rules” 
(Eyal, 2019, p. 31) and possess an ability to explicate in line with these 
rules and within this knowledge system on issues familiar to them and on 
new challenges and problems. In short, according to Eyal (2019), exper- 
tise is a “historically specific way of talking” and “doubly external,” mean- 
ing that the expert status and its disciplinary knowledge and practice are 
derived independently of the expert (p. 31), and it is constructed in net- 
works and connections between individuals (Eyal, 2013). 

In today’s world, expert knowledge is often expected to be based on 
scientific knowledge or, as Holst and Molander (2017, p. 238) noted, on 
knowledge that is validated by scientific norms and procedures. Although 
to be recognized as an expert one must operate according to scientific 
norms of knowledge production, knowledge and expertise for the use of 
policy making are not necessarily provided only by academics. One can 
gain an expert position through practical experience as well. For example, 
civil servants with extensive work experience in a certain field may have 
gained knowledge and skills that are considered “expertise” in that field 
(Holst & Molander, 2019; Krick et al., 2019). 

Due to complexity and global interconnectivity, governments fre- 
quently refer to expertise to legitimize and frame political decisions 
(Moore, 2017, p. 3). Legitimization can be the most significant motive 
for choosing the evidence, particularly in the case of unpopular reform 
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proposals (Maasen & Weingart, 2009; Steiner-Khamsi, 2003, 2004). As 
a legitimization instrument, numbers have become particularly seduc- 
tive. Their power lies in their appearing neutral, apolitical, and objective. 
Still, any political narrative can be attached to them (Stone, 2016). 
Another appealing feature of numbers and quantitative indicators is that 
they make complex realities and processes appear simple and comparable 
(Espeland, 2015, p. 61). 

The increasing demand for evidence-based policy making and expert 
knowledge based on numbers is a familiar phenomenon in education 
policy making as well. Wiseman (2010, p. 1) stated that evidence-based 
policy making in education has become particularly popular since educa- 
tion became closely tied with the economic, social, and political status of 
modern nation-states. This has led to raised expectations related to edu- 
cation system outcomes. Measurable results are expected as a revenue for 
the public expenditure invested in education. Evidence-based policy 
making, according to Wiseman (2010, p. 1), rests on two underlying 
assumptions: (1) education is abstract and universal and (2) empirical 
evidence is an efficient indicator of knowledge and learning. These two 
notions combined have created the belief that one can find what works 
well in one context and apply it to another (Wiseman, 2010). 

The popularity of international large-scale student assessments stems 
from this kind of thinking and supports its further development (Gorur, 
2016). The knowledge and evidence used in education policy making are 
frequently, in fact, globally comparative and quantitative. Prior research 
demonstrates the influence international organizations, such as the 
OECD and its international assessment instruments (e.g., PISA), have 
on nation-states policy making in the field of education (e.g., Costa, 
2011; Grek, 2009; Nieman & Martens, 2018; Sellar & Lingard, 2013; 
Takayama, 2008; Waldow & Steiner-Khamsi, 2019). The views research- 
ers take on this development vary. There are those who praise traveling 
reforms as proof of policy learning and implementation of best practices, 
and those who view this development as a sign of global players imposing 
their own standards on national governments (Steiner-Khamsi, 2012, 
pp. 3-4). The most critical voices have raised concerns about high-level 
strategic thinking being outsourced from the national to the 
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transnational level and to supranational expert organizations (Lawn et al., 
2011, p. 18). 

Both Doreen Massey (2005) and Saskia Sassen (2007, 2013) proposed 
that we view national or local and international or global not as separate 
or layered. They argued that both local and global are constantly under 
construction and that the space of global/international is, in fact, pro- 
duced within the place of local/national. Massey (2005, p. 9) suggested 
thinking of space as something constituted in both local and global and 
in the interaction between these two and as something fluid and in a 
constant state of becoming. Applying these thoughts to the field of com- 
parative education, Larsen and Beech (2014, p. 85) claimed that much of 
the previous research on education transfer was based on a static view of 
transfer, "as if ideas are produced in one site and then received in another 
context." On the contrary, Larsen and Beech (2014) described education 
transfer as the "movement of educational knowledge across space" (p. 76) 
and suggested that comparative education research adopt a theoretical 
framework that focuses on researching networks and connections within 
which educational knowledge is constructed and flowing. Returning 
back to expertise as a historically specific way of speaking and constructed 
in interactions and interrelations as proposed by Eyal (2013, 2019), he 
further proposed focusing on networks as well. He claimed that to under- 
stand the way expertise is constructed and functions, we need to shift our 
attention from individuals to networks. Inspired by these discussions, we 
built our analysis on interactions between local and global and on net- 
works of knowledge, individuals, and organizations. 


Curriculum Process in the National 
and International Context 


Our case in point, the curriculum reform of 2014, was framed by the 
national policy process. The main actors of the process were the Finnish 
government and the National Agency for Education, a government 
agency working under the auspices of Finland's Ministry of Education 
and Culture. The Finnish government politically steered the process, as it 
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is responsible for the general objectives of the National Core Curriculum 
and the distribution of lesson hours. This is legislated in a government 
decree on objectives and distribution of hours, which also delegates the 
power to decide on the core curriculum to the National Agency for 
Education (Valtioneuvoston asetus 422/2012). The decree was issued 
based on a white paper. The political nature of the process became appar- 
ent when the last National Core Curriculum reform was rebooted after 
the draft of the general objectives and distribution of lesson hours was 
met with objections inside the center-right coalition government in 2010 
(Siekkinen, 2017). Subsequently in 2012, the right-left coalition govern- 
ment that followed began work on National Core Curriculum reforms. 
The main governance organ in the process, the National Agency for 
Education, is responsible for curriculum planning and works together 
with the Ministry of Education and Culture to determine specific objec- 
tives and content for subject-specific and cross-curricular themes (Kujala 
& Hakala, 2020). In this work, evidence was obtained from green papers 
and experts. Open consultations can also be part of the process. The 
National Agency for Education was responsible for coordinating a coop- 
erative process with a broad selection of stakeholders to draft the latest 
curriculum (Kujala & Hakala, 2020). For instance, the working group 
for green paper 1, the 2010 document “Basic Education 2020,” orga- 
nized five seminars specifically targeted to certain expert groups; received 
feedback from two-thirds of the Finnish municipalities; heard from 
researchers, experts, and stakeholders on different occasions; and consid- 
ered the survey responses of over 60,000 children and young people in 
which they shared their thoughts on what was good about the school as 
it existed and what changes they would like to see in the future. 
Following completion of the final draft of the latest curriculum, on 22 
December 2014 the National Agency for Education issued the new cur- 
riculum. Since August 2016, grades 1-6 have followed the new curricu- 
lum; grades 7-9 began implementation in 2017—2019. Representatives 
for the National Agency for Education introduced the new curriculum as 
built around “competences needed in society and working life" that 
aimed to change "the content of teaching, pedagogy and school practices" 
(Halinen et al., 2014). According to Uljens and Rajakaltio (2017), the 
new curriculum follows the competence-based idea and key 
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competencies promoted by the OECD. However, the competence orien- 
tation is less radical in comparison to curricula of other countries, such as 
Norway (Molstad & Karseth, 2016; Sivesind et al., 2016). 

The policy development process for Finland’s National Core 
Curriculum has been in flux. The main structural change occurred in the 
1990s. During a period of decentralization, legislation was changed to 
increase the autonomy of municipalities on education issues (Kuntalaki 
365/1995, 1995; Laki peruskoululain muuttamisesta 707/1992, 1992). 
School inspections were gradually abolished (Varjo et al., 2016), and leg- 
islative restrictions on school choice were again tightened in 1998 
(Ahonen, 2003, pp. 180—192; Seppänen, 2006, pp. 66-71). As a result, 
the curriculum has a dual character in steering. On one hand, the 
National Core Curriculum is the main content steering instrument in 
Finnish comprehensive education policy, and schools are legally bound to 
follow it. The Basic Education Act (628/1998, $30) states that "an 
enrolled pupil shall be entitled to teaching according to the curriculum." 
In practice, the National Core Curriculum obligates the provider of edu- 
cation (in most cases, municipalities) to include its central aims as part of 
the educational program. On the other hand, broad degrees of freedom 
are at play in the implementation of the curriculum. The providers of 
education and schools can draw on the National Core Curriculum to cre- 
ate their own curricula. Importantly, apart from formal complaints, no 
direct methods for monitoring the implementation of the curriculum 
exist, which leaves much autonomy for teachers and schools. 

The recurring debate in comparative education addressed in this vol- 
ume is the extent to which education is national or international. The 
degree of divergence or isomorphic convergence of national policies (e.g., 
Meyer et al., 1997; Steiner-Khamsi, 2010, p. 332) and the calls for 
reshaping or deconstructing the understanding of these nation-based cat- 
egories (e.g., Kettunen, 2011; Robertson et al, 2002; Werner & 
Zimmermann, 2006) are examples of this debate. The central questions 
circle around defining the role of the state and whether it is a central 
player in the globalizing world. The corpus of Finnish research on influ- 
ences in education policy is characterized by the tension between these 
two arguments existing at the same time but pulling in opposite direc- 
tions. The analysis of these influences in education policies has focused 
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on the relation of Finland to international organizations. The first argu- 
ment sees the international influences important in shaping the national 
policies, finding the OECD especially important in this role; however, no 
clear agreement has been reached on how and what influence is chan- 
neled. The second argument supports the state-centeredness of Finnish 
policy making. 

In relation to the argument supporting the influence of international 
organizations, interviews with top officials in the Ministry of Education 
and Culture indicate the close relationship the Ministry shares with the 
OECD (Niukko, 2006). In higher education, major reforms have fre- 
quently been preceded by an OECD investigation (Kallo, 2009). In some 
instances, the influence is thought to be rather direct, as Rinne and 
Simola (2005, p. 16) pointed out that quality discourses are "directly 
from the arsenal of the EU and the OECD," and Kauko and Varjo (2008) 
have observed Finland riding in the OECDS slipstream. Then again, 
Kallo (2009, p. 357) understood the influence to be more epistemic: the 
OECD forms an epistemic community, the power of which is derived 
from deeply rooted networks. Moisio (2014) noted that in higher educa- 
tion policy making, Finland has resorted to a “policy spin,” where national 
goals are fed back into the Finnish system via the EU. Naumanen and 
Rinne (2008) demonstrated that the national goals are not always far 
from European or OECD objectives. Therefore, the first argument is 
characterized by the main deduction of a high-functioning and rather 
technically flavored network, ending in influences floating to national 
policy space. 

In relation to the argument for state-centeredness, a historical over- 
view reveals how intensively the state became involved in education. In 
simplified terms, the grand change in nineteenth-century education in 
Finland was its secularization from the church-led system, while the pri- 
mary change in the twentieth century was the nationalization and munic- 
ipalization of education (Joutsivuo, 2010; Leino-Kaukiainen & 
Heikkinen, 2011). The state-centered tradition in Finnish education 
politics is related to the strong state institutions created in the post-war 
period that are responsible for education and to the strong dependency 
that educational institutions and providers have to them. State- 
centeredness is aligned with what has been recognized as the ideal of the 
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universalistic Nordic welfare state (Esping-Andersen, 1990). The build- 
up of the education system was a state-centered process in which compre- 
hensive schools were planned as a large societal project to support equality 
and economic growth (e.g., Ahonen, 2003), while at the same time pub- 
lic higher education was subject to regionalization and massification (e.g., 
Lampinen, 2003). Simultaneously, the state institutions grew in impor- 
tance. The post-war “post-office size" (Kivinen et al, 1990, p. 39) 
Ministry of Education and Culture budget sector became the third larg- 
est among the ministries. The strong role of the state was slightly reorga- 
nized at the time of the global and Nordic (Dovemark et al., 2018) 
management reforms, where public governance in Finland was reformed 
to a strategic and managerial style (Autio, 1997; Temmes, 1996). As part 
of the international trends, the 1990s brought a move toward decentral- 
ization and deregulation, which changed the steering system dramatically 
and gave responsibility to the municipalities (Simola et al., 2013). 
However, the Ministry of Education and Culture is still recognized as a 
central if not the most central power hub in Finnish education policy 
making. Research has documented its role as a bureaucratic-led and inde- 
pendent actor rather than a politically steered organization (Kivinen 
etal., 1990, p. 103; Lampinen, 2003, pp. 162—200; Lehtisalo & Raivola, 
1999, pp. 122-123). However, research has also identified that the gen- 
eral national policy steering tends to supersede the education-based pol- 
icy steering signals (Kallunki et al., 2015; Seppänen et al., 2019), which 
does not diminish the argument of a state-centered system but, rather, 
supports it. In sum, the second argument sees the long-term growth of 
the state system. 

The analysis of the policy process in Finland has revealed the links 
between the state bodies and the international organizations. The two 
partly opposing lines of argumentation provided by previous research on 
Finnish education policy and politics gives a rich picture of a system that 
traditionally has been strongly state led, yet has also become part of a 
global policy space and network from which influences float to national 
policy space. However, this picture is still rather dichotomous and at least 
partly based on the idea of ideas flowing from one level (international 
and global) to another (national or local), and previous research has 
revealed also more interactive processes (Centeno, 2017). Our aim is to 
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surpass this dichotomy and to complement the picture by focusing on 
networks of evidence and expertise, as Larsen and Beech (2014) and Eyal 
(2013, 2019) proposed. In the process we aim to discover, along the line 
of Massey (2005) and Sassen (2007, 2013), how global is constructed 


in local and vice versa. 


Research Design 
Research Question and Methods of Analysis 


In this chapter we set out to explore how evidence was used in Finland’s 
2014 reform of the National Core Curriculum and whose evidence was 
most highly valued in this process. As the starting point for our method- 
ological approach, we chose to apply Paul Cairney's (2016) definition of 
evidence as "an argument or assertion backed by information (p. 3)". We 
share the view that Gita Steiner-Khamsi presented in Chap. 2 that the 
importance of examining bibliometric references is crucial since these 
references in the policy documents are used to "provide legitimacy to the 
evidence which the author has provided in the document.” Hence, refer- 
ences can be seen as validation of evidence, so we applied a bibliometric 
network analysis as our first method in this chapter. In the bibliometric 
network analysis, we focused on the 677 bibliometric references used in 
the ten core policy documents that constituted our database. These refer- 
ences were analyzed with the software programs UCINET and Netdraw, 
which generated descriptive statistics and an illustrative figure of knowl- 
edge networks. The statistics were used to examine what kinds of evi- 
dence (location of the publication; type of publication) and expertise 
(author of the publication) were used in the reform. The visualization of 
the networks was used to illustrate the political process, the knowledge 
network, and evidence base of the reform. 

We double-checked and complemented these findings with a content 
analysis of the ten documents selected for investigation. We inspected the 
ten core documents, looking for any references to the OECD and PISA 
that appeared within the text but were not references to actual sources 
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nor included in the reference list. Finally, we examined each of the most 
cited OECD documents and identified the policy documents in which 
they were used. 

To determine whose evidence was most highly valued in this reform, 
we additionally investigated the organizations that employed the most 
prominent authors at the time they authored these documents. The top 
18 of the most cited individual authors were selected and their literary 
works and the year of publishing that were found in the database were 
listed chronologically, after which their employers at the time that they 
wrote each text were researched using a variety of sources, such as other 
publications from the same year that had the author's position and orga- 
nization cited, social media profiles, publications and newsletters from 
the employing organizations, newspaper articles, worker profile pages of 
universities and other organizations, and even biographies in the case of 
some of the more experienced authors. In addition, we analyzed the titles 
of all 677 publications in our sample and singled out the titles of the 
publications of the most cited authors for closer analysis. 


Selection of Data 


As there is not a universally agreed upon concept of a white paper (WP) 
or a green paper (GP), an interpretation was made that a WP constitutes 
a draft for new legislation, while GPs provide background information. 
Our focus of analysis for the main document was a WP produced by the 
ministry-assigned working group. The WP “Future Basic Education" 
identified the general national objectives, presented a proposal for 
renewed distribution of lesson hours, and suggested the necessary decree 
changes for the Government Statute on the National Education 
Objectives Referred to in the Basic Educational Act and the Distribution 
of Lesson Hours in Basic Education (Valtioneuvoston asetus 422/2012, 
2012). From the in-text citations typical of the Ministry of Education 
and Culture documents, several documents were discovered, nine of 
which were identified as GPs suitable for analysis. 

Our criteria for determining relevance primarily consisted of three fac- 
tors: (1) the document provided relevant information and suggestions for 
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the curriculum reform, (2) the document contained a list of references or 
references in footnotes, and (3) the document was related to the WP. The 
input of data was completed in two stages: the primary input stage, dur- 
ing which data from the selected sources were coded, and the secondary 
stage, during which the coded data were cleaned and corrected. Some 
subsequent data-cleaning was completed later when the need arose, but 
those changes were minor compared to the second stage and did not alter 
the original findings. 

The coding process followed the original plans for creating a compa- 
rable database. In the Finnish coding process, some clarifications had to 
be made, especially when categorizing the documents as “book,” “report,” 
“journal article,” or “government-issued report.” The categorization of 
document types was problematic for two reasons: the National Agency 
for Education conducts plenty of scientific research on its own and in 
conjunction with certain Finnish universities, most of which is then pub- 
lished under the agency’s name, making it difficult to distinguish the level 
of government involvement at times. Second, many faculty in Finnish 
universities have their own publication series where studies conducted by 
the members of the faculty are presented, often in a book form. Many of 
these studies are peer-reviewed, making the exact categorization of the 
publication series difficult. These issues were solved as follows: any stud- 
ies published by the National Agency for Education were coded as a 
“government-issued report,” and the publication series were coded as 
either a “book” or a “report,” depending on the form of the publication, 
as they are not academic journals in the strictest sense. 

Listed in Table 5.1 are all the documents selected for the analysis. All 
the selected documents were written in Finnish; their English transla- 
tions follow the document names in parentheses. Throughout the rest of 
this chapter, tables with Finnish words and names have their translations 
or explanations in parentheses for the ease of readability. 
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Table 5.1 Documents chosen for analysis 


DocID Document title 


66 (WP) Tulevaisuuden perusopetus [Future Basic Education] 

1(GP)  Perusopetus 2020—yleiset valtakunnalliset tavoitteet ja tuntijako 
[Basic Education 2020: Common National Aims and Division of 
Teaching Hours] 

2(GP) Opinto-ohjauksen arviointi perusopetuksessa, lukiossa ja 
ammatillisessa koulutuksessa sekä koulutuksen siirtymávaiheissa 
[Evaluation of Student Counseling in Basic Education, Upper 
Secondary Schools, Vocational Education, and in Transition Phases 
of Education] 

3 (GP) Onko laskutaito laskussa? Matematiikan oppimistulokset peruskoulun 
päättövaiheessa 2011 [Are Mathematical Skills in Decline? Math 
Learning Results at the End of Basic Education in 2011] 

4 (GP)  Aihekokonaisuuksen tavoitteiden toteutumisen seuranta-arviointi 
2010 [Evaluation of Achievement of Overarching Education Goals 
2010] 

5 (GP) Opettajat Suomessa 2010 [Teachers in Finland 2010] 

6 (GP) Esi- ja perusopetuksen opetussuunnitelmajärjestelmän toimivuus 
[Evaluation of the Curriculum of Pre-School and Primary Education] 

7 (GP) Liikunnan oppimistulosten seuranta-arviointi perusopetuksessa 2010. 
Koulutuksen seurantaraportit 2011:4 [Evaluation of Learning Results 
in Physical Education 2010. Educational Evaluations 2011:4] 

8 (GP) Luonnontieteiden seuranta-arviointi [Evaluation of Natural Sciences] 

9 (GP) Historian ja yhteiskuntaopin oppimistulokset perusopetuksen 
päättövaiheessa 2011 [Evaluation of Learning Results in History and 
Social Studies at the End of Basic Education 2011] 


Notes: English translation of document names in parentheses. WP = White Paper; 
GP = Green Paper 


Results 
Evidence Base of the Reform 


The evidence base of the reform consisted of 677 referenced documents. 
Based on prior research, both international and Finnish, we expected to 
see significant use of international sources for policy evidence. Based on 
the prominence of the OECD in producing quantitative comparative 
data, the organization was anticipated to be especially prominent, and 
the evidence in this policy process was expected to be extensively drawn 
from PISA and other large-scale student assessments. On the contrary, 
however, the Finnish data indicate a strong state involvement and 
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concentration of expertise in state organizations, namely, the Ministry of 
Education and Culture (that produced the WP), and especially, the 
National Agency of Education (that produced or commissioned the GPs 
in our sample). Table 5.2 depicts the distribution of references according 
to the location of publication (domestic, regional/Nordic, international) 
and the type of publication (report, book, journal article, governmen- 
tal, other). 

Of all references included in the ten policy documents, 7696 are 
domestic, that is, published in Finland, and 2296 are international, while 
only 1.696 were published in other Nordic countries, or regionally. In 
three of the ten policy documents, all references used are domestic. An 
analysis of the type of publications reveals that very little scientific evi- 
dence was used in the process. The percentage of journal articles is very 
low, only 9.4596 in total. Almost no scientific evidence in the strictest 
sense of the definition was used in the WP (document ID 66), as the 
percentage of peer-reviewed academic journal articles is 0%. However, it 
is important to remember that, as explained previously (see "Selection of 
Data” section), the categorization of publications in the Finnish case was 
challenging. On one hand, the National Agency of Education publishes 
books that are written or co-authored by universities and researchers and 
comply with scientific norms. On the other hand, universities publish 
book series that are not peer-reviewed in the strictest sense but, neverthe- 
less, are academic publications. As we chose to follow the strictest possi- 
ble interpretation of peer-reviewed academic publications (categorized as 
“journal articles”), any studies published by the National Agency for 
Education were coded as “government-issued report,” and the publica- 
tion series were coded as either a “book” or a “report,” depending on the 
form of the publication, as they are not academic journals in the strict- 
est sense. 

The network analysis reveals that the evidence base of the WP (in the 
bottom left hand corner of Fig. 5.1, document ID 66) to a great extent is 
based on the evidence base of GP 1 “Basic Education 2020,” shown in 
the middle of the figure. In fact, the evidence base of the GP appears to 
be more central and versatile in this reform than that of the WP itself. It 
can also be concluded that in this reform, GP 1 “Basic Education 2020” 
serves as an “intermediary” (Lubienski, 2019), as it connects the 
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Fig. 5.1 Complete network structure. (Notes: Regional: gray, domestic: white, 
international: black; source: Circle; node size=in-degree centrality) 


knowledge network of the WP to that of the other GPs. The link strengths 
are illustrated in Fig. 5.2. 

The WP (66) is linked only to two other documents, GP 1 “Basic 
Education 2020" (1) and the GP 6 (6) *Evaluation of the curriculum of 
pre-school and primary education." The link between the curriculum 
evaluation GP and the WP is rather weak, indicating they share some of 
the same sources but not many. However, the link between the "Basic 
Education 2020" GP and the WP is strong, indicating they share largely 
the same sources and, hence, the same evidence base. To summarize, the 
evidence base of this reform appears to be predominantly domestic, and 
the WP and GP 1 share a significant portion of their evidence base. 
Furthermore, GP 1 serves as an intermediary and is the most central 
document in the network. 
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Fig. 5.2 Source document network. (Note: Link strength is based on the number 


of references shared by two sources) 


Table 5.3 Most cited publishers (top ten) 


Publisher 


Opetushallitus [National Agency for education] 

Opetus- ja kulttuuriministerió [Ministry of Education and Culture] 
Jyváskylàn yliopisto [University of Jyvaskyla] 

Helsingin yliopisto [University of Helsinki] 

Taylor & Francis 

PS-Kustannus (Publisher) 

Organisation for Economic Co-operation and Development [OECD] 
WSOY (Publisher) 

Valtioneuvosto [Finnish Government] 

Tilastokeskus [Statistics Finland] 

Valtion painatuskeskus [State Printing Center] 

TOTAL 


Whose Evidence Is Most Highly Valued? 


Count 
170 


As discussed previously, the evidence base of this reform appears to be 
overwhelmingly domestic. A closer look at the publishers illustrates that 
the evidence base is not just domestic, but it is also exceedingly state- 
centered. Table 5.3 provides a list of the top ten most cited publishers. 
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Of the 395 publications on the most cited publishers list, 4396 (170) 
were published by the National Agency of Education, and 14.596 (57) 
were published by the Ministry of Education and Culture. Therefore, of 
the documents published by the top ten most cited publishers, 57.596 
(227) were published by one of these two government organizations. If 
the documents published by the Finnish government (9) and the State 
Printing Center (8) are added, the total amount published by a state/ 
government organization rises to 244 publications, which is 62% of the 
documents published by the top ten most cited publishers. This indicates 
a strong state involvement in the curriculum reform process and demon- 
strates that the evidence base of this reform was national, state-centered, 
and to a great extent, self-referential in nature. 

On this list, the University of Jyvaskyla ranks third and the University 
of Helsinki ranks fourth. As described in Chap. 11 (Ydesen, Kauko, 
Magnúsdóttir), these two universities hold a specific and special position 
in Finnish education policy making. The Finnish Institute of Educational 
Research at the University of Jyväskylä and the Centre for Educational 
Assessment at the University of Helsinki have been, in turns, contracted 
by the Ministry of Education and Culture to implement PISA. The 
Finnish Institute of Educational Research has also been responsible for 
the OECD Teaching and Learning International Survey (TALIS). They 
are both main hubs for OECD data expertise in Finland. In this particu- 
lar curriculum reform, they were also among the main publishers of the 
evidence used in the reform. They are the only universities in this top ten 
list, indicating that it is not so much the scientific evidence but the evalu- 
ation expertise, and in particular, the OECD data expertise, that was 
valued in this reform. Hence, it seems that the evidence used in this 
reform is domestic and state-centered, and it is particularly assessment 
and evaluation results that were valued as evidence. A closer look at the 
most cited authors further strengthens this argument (see Table 5.4). 

As on the list of the top ten most cited publishers, the National Agency 
of Education and the Ministry of Education and Culture occupy the two 
top positions on the list of the top ten most cited authors. Both are refer- 
enced as authors almost three times as often as, for example, the 
OECD. Even individual authors, like Jakku-Sihvonen, a former employee 
of the National Agency of Education, are referenced almost as many 
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Table 5.4 Most cited authors 


Author Count 


Opetushallitus [National Agency for Education] 45 

Opetusministerió/Opetus- ja kulttuuriministerió [Ministry of Education 38 
and Culture] 

Organisation for Economic Co-operation and Development [OECD] 15 

Lappalainen, H.-P. 13 

Jakku-Sihvonen, R. 12 

Uitto, A. 11 

Eduskunta [Finnish Parliament] 9 

Nupponen, H. 

Vuorinen, R. 

Lavonen, J. 

Mattila, L. 

Telama, R. 

Valijarvi, J. 

Valtioneuvosto [Finnish Government] 

Kupari, P. 

Silverstróm, C. 

Junttila, N. 

Kasurinen, H. 

Lairio, M. 

Metsámuuronen, J. 

Arinen, P. 

Atjonen, P. 

Heikinaro-Johansson, P. 

Houtsonen, L. 

Kari, J. 

Linnakylä, P. 

Mehtálàáinen, J. 

Nummenmaa, A. R. 

Puhakka, E. 

Rantanen, P. 

Valkonen, S. 

Vauras, M. 


Note: Cutoff point: minimum five times 


U1 Ul Ul U1 Ul U1 Ul Ul Ul Ul Ul UI O) O) O» O: N N OO OO OO OO OO (tO WHO 


times as the OECD. Such individual authors can be seen as top experts 
in this particular reform but possibly also in the field of Finnish educa- 
tion policy in general. A closer look at the background of these top experts 
reveals that many of them were, at the time they authored the documents 
that were referenced in this policy process, employed by the National 
Agency of Education (5) or by the two universities that traditionally work 
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closely with the Agency—the University of Jyvaskyla (6) and the 
University of Helsinki (6). 

This demonstrates that the Ministry of Education and Culture and the 
National Agency of Education used, for the most part for this reform, 
evidence produced within their organizations or by the organizations 
they have a close relation with. This indicates that the policy evidence in 
this reform was largely self-referential and the policy process firmly 
steered by national government organizations. These findings also high- 
light the strong expert position and power of the National Agency of 
Education in the field of education policy and politics in Finland. 

On the basis of these findings, it would be easy to conclude that policy 
making in Finland is, indeed, state-centered, the evidence base predomi- 
nantly domestic, and the influence of international evidence and transna- 
tional organizations trivial. However, a closer look at the policy documents 
in our sample tells a slightly different story. 

Firstly, although the OECD amounts to only 3% of the publications 
in the ranking of the top ten most cited publishers, and in the ranking of 
most cited authors it has 14 publications amounting to only 0.05%, it is 
important to note in which of the ten core policy documents these 
OECD documents were used. 

The OECD references appear mainly in the WP and GP 1 “Basic 
Education 2020.” The GP 1 was originally intended to be a WP, as 
explained in more detail earlier in "Curriculum Process in the National 
and International Context" in this chapter. Ihough the OECD refer- 
ences are not used in all of the policy documents in our sample, and the 
OECD documents are cited far less frequently than domestic references, 
OECD references are used in the policy documents that were most influ- 
ential in terms of policy design and selection of the actual content of the 
curriculum reform. Thus, one can state that the OECD was, in fact, a 
significant component in the evidence base for this reform and that the 
OECD evidence was considered valuable. 

Secondly, earlier research as referred to previously in this chapter has 
demonstrated the influence of large-scale student assessments (e.g., PISA) 
and the transnational organizations behind them in the national level of 
education policy making. These organizations and their standardized 
tests promote, in particular, a focus on skills, student achievements, and 
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evaluation of outcomes. In our sample, all GPs, apart from GP 1 “Basic 
Education 2020,” are evaluations of some sort. Five are national assess- 
ments of learning outcomes. Moreover, 77 documents in our database 
(according to their titles) address topics related to evaluation or are evalu- 
ation reports of different subject areas and skills. It does seem that what 
was valued most as evidence in this reform are evaluations of skills and 
outcomes, indicating the evaluation and assessment culture promoted by 
the OECD does have an influence in Finnish policy making as well. 

Thirdly, a content analysis of these eight GPs reveals that references to 
PISA are sometimes used in the text, even though no actual source is 
given as a reference in the text or in the list of references. On the other 
hand, the references may be to publications of Finnish experts and not 
directly to the publications by the OECD. This can be seen in GP 3, an 
evaluation on mathematics skills. Paragraph 1.3 in this report is dedi- 
cated to PISA results. In the rest of the report, the OECD is mentioned 
five times in the whole document, “OECD-countries” are mentioned five 
times, and PISA receives 24 mentions— 19 in the text and 5 in the refer- 
ences. PISA is used as evidence within the text throughout this national 
evaluation report, but the references used pertaining to PISA results or 
the OECD are domestic references. In other words, the references come 
from three domestically produced and published documents, one by the 
Ministry of Education and Culture and two by the PISA experts of the 
University of Jyvaskyla. 

In addition, more findings must be considered when discussing the 
role and influence of the OECD and the evaluation culture it promotes. 
A significant portion of the publications of the most cited authors are 
national evaluation reports or reports written based on either national or 
international assessments. For instance, authors Lappalainen and Mattila 
are very high up on the list of most published authors. The six publica- 
tions for which Lappalainen was the first or only author concern the 
national evaluations of learning outcomes in Finnish language as the 
mother tongue and in literature. The publications with Mattila as first 
author discuss national evaluations of learning outcomes in mathematics. 
This indicates that evaluation expertise and evidence especially were val- 
ued in this reform. Finally, one specialized form of this kind of expertise 
is the ability to interpret and translate OECD data so that it fits the 
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national policy context and needs. Among the most cited authors are for 
instance, Valijarvi, Kupari, and Arinen, each of whom belongs to this 
group of experts, their cited publications being predominantly Finnish 
language reports on PISA results. 


Discussion 


Our analysis of the evidence used in the curriculum reform 2014 reveals 
a strong tendency to use not only domestic evidence, but evidence pro- 
vided by the two most powerful state organizations in the landscape of 
Finnish education policy making: the Ministry of Education and Culture 
and the National Agency for Education. These two organizations hold a 
strong, legally established position as the most prominent expert organi- 
zations in Finnish policy making in terms of comprehensive education 
and the National Core Curriculum. In the curriculum reform of 2014, 
they produced most of the nine core policy documents according to their 
mandate; the remaining documents were commissioned by the National 
Agency for Education. The working groups consisted, to a great extent, 
of the civil servants of either the Ministry of Education and Culture or 
the National Agency for Education, although the working group for 
"Basic Education 2020" from 2010 included representatives of main 
political parties and the most prominent stakeholder organizations as 
well. In addition, the bibliometric analysis of the cited references demon- 
strates that the working groups behind these policy papers chose pre- 
dominantly to include evidence coming from within these two 
organizations as references in their reports, or alternatively, documents 
provided by organizations that have a strong and extended relationship 
with these two organizations as the contracted institutions for imple- 
menting PISA in Finland. 

Our findings suggest, however, that there is more to the story. Though 
we found that the evidence used in the curriculum reform of 2014 was 
predominantly domestic and self-referential, this finding does not paint a 
correct picture of the influence of transnational actors, particularly the 
OECD. The content analysis revealed that PISA was referred to in the 
key policy documents, but sometimes without a bibliometric reference to 
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the OECD or PISA. In addition, a significant part of the references cat- 
egorized as domestic in our sample were, in fact, reports on the PISA 
results written by Finnish authors and published in Finland. It seems the 
OECD data and, particularly, PISA indicators were, indeed, used as evi- 
dence in this reform, but this influence was filtered through domestic 
experts and expertise so that in the bibliometric analysis it appears mostly 
as domestic evidence. 

The OECD data in this reform were especially used in the two most 
influential papers in terms of actual policy design: the WP and GP 1, 
which, when drafted, was intended as a WP but was discarded after the 
political power changed from the center-right to right-left. This indicates 
that the importance of the OECD and its international assessment instru- 
ments cannot be measured merely by looking at the number of references 
used in the policy documents. In the case of the curriculum reform of 
2014, it seems that the OECD data and references came into play par- 
ticularly in the policy documents that carried the most weight in the 
official policy design. This may indicate that the OECD evidence was 
used primarily to legitimize the policy recommendations and design, but 
this theory would need to be researched further. Our results, however, 
support the ideas of Deborah Stone (2016) and Wendy Espeland (2015) 
that numbers appear objective and apolitical, making them particularly 
appealing to policymakers. Sometimes these numbers from different 
evaluation reports come from within the local and national context and 
other times from the global and international context. Nevertheless, the 
hierarchization of evidence Gita Steiner-Khamsi discusses in Chap. 2 is 
undeniably evident in our example of an education reform as well. 

In terms of expertise, what seemed to count in the main reform docu- 
ments was, in fact, expertise in evaluation and assessment. '[he evidence 
that was valued was not necessarily scientific, but it was predominantly 
empirical. Though we did not specifically focus on narratives attached to 
numbers, it became clear to us that the preference in this reform was for 
a certain kind of evidence. The information used was to a large extent 
empirical evaluation data, partly stemming from the OECD and its PISA 
indicators. This is very much in line with Wisemans (2010) claim that 
the evidence agenda in education is based on an underlying assumption 
that empirical evidence is an efficient indicator of knowledge and 
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learning. However, it was often only after the empirical data was filtered 
through the expertise of the core national actors that the data became 
policy evidence. The international data were many times filtered through 
Finnish officials’ publications or by Finnish institutions. 

Holst and Molander (2019) claimed that expertise does not necessarily 
have to be scientific. It can also be built on professional knowledge gained 
through long working experience in the field. The experts providing 
(most cited authors) and selecting (members of the working groups and 
the most cited authors) evidence in this reform were both scientists and 
civil servants with extensive experience in the field. What they frequently 
had in common, however, was expertise in assessing learning outcomes or 
in interpreting the results of large-scale student assessments. Our findings 
indicate that the ability to select, interpret, and translate this kind of data 
so that they fit the national context is what gives specific national actors 
the authority to speak with the authority of an expert. Most had expertise 
in explicating evaluation data, either national or international. Hence, 
the historically specific way of talking that Eyal (2019) referred to appears, 
in this case, to be the ability to speak numbers. The double externality of 
expertise (Eyal, 2019) in this process is evident in the fact that this par- 
ticular form of expert talk was combined with the already established 
expert position of the individual authors as representatives of the most 
prominent power hubs in Finnish education policy making and politics. 
In fact, our results illustrate that expertise in education policy in Finland 
is highly concentrated in a few organizations that work closely together. 
It is this expert position that appears to give these organizations and their 
most prominent experts rather sovereign power to determine what con- 
stitutes policy evidence. 

We began our chapter with a discussion on the role of the state in 
Finland's education policy making and whether the influence of interna- 
tional organizations has increased to the point that high-level expertise 
(and national decision-making along with it) is outsourced to the trans- 
national level. Our research has demonstrated that education policy mak- 
ing, at least at the comprehensive education level in Finland, is primarily 
in the hands of two state organizations—the Ministry of Education and 
Culture and the National Agency for Education. We found no evidence 
of the transnational level overriding national expertise. In fact, the 
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national players appeared to be a powerful filter between the global and 
local. Although part of the evidence was produced on the transnational 
level (particularly by the OECD), the selection of evidence for policy 
decisions was carried out on the national level and by domestic experts. 
Also, part of the expertise of the national players consisted of explicating 
international evidence so that it fit the national context. The case we 
examined indicates that the international or global has the power to pro- 
duce evidence but the national or local has the power to select the evi- 
dence and adjust it to meet the national needs. In fact, the type of 
evidence appeared to be more important than where the evidence origi- 
nated from (local or global level). Both national and international evi- 
dence were used in this reform, but the key trait of the evidence was that 
it largely comprised empirical evaluation data. Similarly, the ability to 
“speak empirical evidence” seemed to be a key characteristic of the experts 
involved. This could include both domestic and international evidence, 
but the role of national experts in selecting, translating, and possibly even 
modifying the evidence produced by global players remained central in 
the process. This indicates that much like Larsen and Beech (2014) sug- 
gested, in the current era of global education transfer, the layered approach 
of researching comparative education no longer applies. Educational 
knowledge, as proposed by Larsen and Beech (2014), moves across the 
global space and through globally connected experts and expert 
organizations. 


Conclusions 


In this chapter we set out to investigate what kind of evidence the reform 
of the Finnish 2014 National Core Curriculum drew on and whose evi- 
dence was most highly valued in the process. We aimed to answer this 
question with a bibliometric network analysis, complemented with con- 
tent analyses of the ten policy documents in our sample. Our database 
consisted of 677 cited references in these documents. Our starting point 
was the recurring debate in comparative education regarding the extent 
to which education is “national” or “international” and the tension of the 
two main arguments related to this debate in the Finnish research corpus. 
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We built our interpretative framework around the concept of expertise as 
a historically specific way of talking, doubly external, and constituted in 
networks and connections between individuals, as theorized by Gil Eyal 
(2019), and around the discussion of the interplay between global and 
local in comparative education research (Larsen & Beech, 2014) inspired 
by Doreen Massey (2005) and Saskia Sassen (2007, 2013). Our main 
methods were bibliometric network and qualitative content analyses. 

At first glance, the state-centeredness appeared evident, as most evi- 
dence fell into the category of "domestic" in the bibliometric network 
analysis. A closer look revealed that reform was, in fact, based on both 
domestic and international evaluation evidence. The expert power 
throughout the process was, nevertheless, firmly held in the hands of a 
rather small, domestic network of experts. Their power appears to lie 
mainly in two areas: (1) the power to select the information relevant for 
policy evidence (as evidence is defined by Cairney, 2016) and the ability 
to explicate empirical evaluation data to cater domestic needs, as our 
findings suggest that "the historically specific way of talking" (Eyal, 2019, 
p. 31) that is needed for recognition of true expertise equates to speaking 
the language of evaluations and numbers; and (2) their relationship to 
either the Ministry of Education and Culture or the National Agency for 
Education, as these experts were either employed by these two most 
influential organizations in Finnish education policy or were working for 
one of the institutions traditionally closely linked to these organizations 
through their special role administrating and interpreting OECD’s PISA 
results. 

Although international evidence was used in the process, our findings 
do not support the most critical predictions that high-level expertise is 
being outsourced to the transnational level. At least in this reform pro- 
cess, the Ministry of Education and Culture stayed firmly in the steering 
wheel as a major hub of education policy expertise in Finland. Even the 
change in political power during the curriculum reform had little influ- 
ence on the reform led by the Ministry and its civil servants. With the 
new government, the curriculum reform stayed its course, and the work 
continued with the once discarded policy document as the basis, regard- 
less of the political changes that surrounded it. Though the process was 
state led, our findings do not support unanimously the claim for 
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state-centeredness, either. Much of the evidence, particularly in the most 
influential policy documents in terms of policy design, came from the 
OECD. Additionally, the expert position in the process was frequently 
based on expertise in explicating transnational evidence. 

All in all, our findings indicate that the two layers of local and global 
are not separate or distinct but interconnected and intertwined. One 
gains its power from the other. Policy evidence may, in fact, function 
both as a legitimation tool for reform and as the source of strengthening 
expert power. In the process it may not be the state itself that is the cen- 
tral player in education policy, although it officially leads it. Much power 
may also be found working through and in the networks of experts. The 
experts are often national but they draw their expertise from the networks 
based on both local and global knowledge and connections. In the glo- 
balized world these networks cannot be traced back to one place but 
instead play out their influence in the global policy space that is con- 
structed only partly within the national. 
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The Irregular Formation of State Policy 
Documents in the Icelandic Field 
of Education 2013-2017 


Berglind Rós Magnusdottir and Jon Torfi Jonasson 


In recent decades, the role of the state in the governance of education has 
been changing, as program evaluations, best practices, and large-scale 
transnational student assessments have visibly shaped policy making in 
the field. This development has been connected to such conceptual 
frameworks as governance by numbers, evidence-based policy, external- 
ization of national policy, scientification, and neo-managerialism 
(Lingard, Martino et al., 2013; Lingard, Sellar et al., 2014; Robertson, 
2016; Dovemark et al., 2018; Steiner-Khamsi, 2016). The purpose of 
this chapter is to explore state policy formation in Iceland by analyzing 
three recent state documents, specifically examining the following: (a) 
their antecedents, procedures, and follow-up; (b) their knowledge pro- 
viders; and (c) the kind of knowledge that is interpreted and recontextu- 
alized into Icelandic documents. 

Early in this process it became clear that the processes guiding educa- 
tion policy formation in Iceland differ from the Nordic protocol that 
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relies on available green papers (GPs) and white papers (WPs) with refer- 
ence lists as the main data sources (Steiner-Khamsi et al., 2020). Thus, 
the first step in the process for this research was to locate documents that 
would fulfill the criteria of WPs and GPs as understood in the research 
project (POLNET). We found three documents that could be considered 
appropriate for the research protocol. In this comparative research proj- 
ect, references are understood as a policy tool to justify or authorize the 
content of reform (Steiner-Khamsi, Chap. 2 in this volume). Additionally, 
we obtained deeper phenomenological understanding of the government 
procedure by analyzing the content of the documents and of qualitative 
interviews with five officials at the Ministry of Education, Science and 
Culture. Due to the scarcity of GPs to examine for the research, we felt it 
was important to conduct the interviews to elicit a clearer picture of the 
knowledge accumulation inside the Ministry. 

The main purpose of this chapter is to explore and compare two vision- 
ary policy documents and one background paper relevant to the reform 
of 2013—2017 to determine what counts as evidence at the stage of 
agenda-setting and policy formulation and identify the main knowledge 
providers according to reference lists. Governmental legal frameworks 
originate from the Ministry of Education, Science and Culture and gain 
legal status in the education system through acceptance from parliament. 
Visionary policy documents, on the other hand, outline what the govern- 
ment hopes to achieve and the knowledge, methods, and principles it will 
use to achieve them. They state the goals of the Ministry of Education, 
Science and Culture and local municipal governments. 

The two visionary policy documents under review in this chapter were 
both processed by the Minister of the Independence Party, Mr. Illugi 
Gunnarsson. The earlier of che two (WP2014) was written and classified 
formally as a WP and is the only document that largely fulfills all criteria 
for a WP (Ministry of Education, Science and Culture, 20143). The more 
recent policy document (WP2017) began as an audit on inclusive educa- 
tion, with background material and guidelines presented by the Ministry 
of Education, Science and Culture, but was adopted as a formal policy 
initiative (European Agency for Special Needs and Inclusive Education, 
2017). Therefore, we also treat this as a WP. The third document reviewed 
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in this chapter is a background paper prepared under the auspices of the 
Ministry of Education, Science and Culture for an intended Organisation 
for Economic Co-Operation and Development (OECD) review on 
resources for the Icelandic schools, which never materialized. Still, the 
document is often quoted in policy discourse, and we treat it as a GP 
(Ministry of Education, Science and Culture, 2014b) and it was cited 
in WP2017. 

‘These texts are important examples of the formal documentation that 
now governs or at least influences Icelandic compulsory education, along 
with laws, regulations, and curricula that are considered of primary 
importance. The document that receives the most attention in our analy- 
sis is the WP from 2014, which was written explicitly and exclusively as 
a policy document. The other two are included to explore whether a very 
different picture is obtained with respect to the substantive underpinning 
of evidence, externalization, and governance procedure. Moreover, it was 
necessary to review these additional texts to obtain a more comprehensive 
understanding of what can count as evidence and of the formal governing 
practice for an issued policy document in the field of education in Iceland 
than could have been gleaned from an analysis of any one single document. 

The chapter begins with an introduction to the Icelandic field of edu- 
cation and governance, followed by methodological considerations for 
this investigation and then a presentation of the results. The empirical 
portion of the chapter is divided into two sections. The first lays out the 
numerical pattern of the database. Its subsections go deeper into the texts 
through a content analysis of the WPs by exploring the actual use of ref- 
erences in the text. The second section deals with the governing practice 
itself within the Ministry of Education, Science and Culture, focusing on 
the agenda setting, the power relations between the politics (Minister) 
and the administration (officials), the given time frame, and choices of 
structure, knowledge providers, and evidence. The qualitative interviews 
on the governance practice were conducted based on the results from the 
first empirical section, specifically, the scarcity of written documents to 
produce a knowledge base for policy making and the numerical pattern 
in the evidence base. 
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Iceland: The Research Field 


To understand the present, we need to provide insight into the past in 
terms of educational reforms and governance concerning the themes of 
the two WPs analyzed—inclusion and literacy. 


A Historical Account of Education Reforms in Iceland: 
Social Justice, Inclusion, and Literacy 


The history of public schools in Iceland is relatively short compared to 
the public school histories of other Nordic countries. 'The first compul- 
sory school reform in Iceland was led by Guómundur Finnbogason, who 
was appointed by the Icelandic government to write a report, a back- 
ground paper, to prepare for the passage of the country's first bill on 
education. He visited progressive schools in Scandinavian countries to 
gather ideas and, in 1903, published his report as a book, as he felt it was 
important to inform not only the government of Iceland but the public 
as well about his ideas (Finnbogason, 1903/1994; Guttormsson, 2008). 
The first act on public schooling was passed in 1907. However, the legally 
unified, compulsory, and comprehensive school did not exist in the law 
until 1946 (Lög um skólakerfi og fradsluskyldu, 1946). The government 
viewed the educational system as "fundamental to the newly found inde- 
pendence of the nation from Denmark which required an educated pub- 
lic capable of running and administering a modern state" (Halldórsdóttir, 
Jónsson, & Magnüsdóttir, 2016, p. 438). 

‘The policy focus on literacy is in line with a strong emphasis on read- 
ing proficiency in Icelandic educational history. Ihe major inspection 
efforts that were undertaken in the 1740s and later in the 1930s had 
their primary focus on reading (Guttormsson, 2008). From the begin- 
ning of public schooling in Iceland in the early twentieth century, reading 
scores in terms of speed were used to track and stream in the early grades 
(Garðarsdóttir, 2001). Indeed achievement in reading has been strongly 
related to the intersections of social class, residence, and gender in Iceland 
(Gísladóttir et al., 2019). As such, immigrant and refugee children in 
Iceland are the “new” disadvantaged group (Garðarsdóttir & Hauksson, 
2011; Hardardottir et al., 2020). Reading proficiency has, for centuries, 
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been the most critical reference point for educational quality, equity, and 
progress. What is new, however, is that the aim for higher achievement in 
reading is now put forward in a visionary policy document (Ministry of 
Education, 20143). 

The 1974 Education Act (63/1974) was progressive in terms of high- 
lighting democracy and inclusion as core purposes, when all children, 
including those with special needs and disabilities, were required to 
attend school (Marinósson & Bjarnason, 2014). This act represented a 
fundamental political change led by a former Minister of Education, Dr. 
Gylfi P Gíslason, a member of the Social-Democratic Party (i. 
Alþýðuflokkur), and his special advisor, Wolfgang Edelstein (Edelstein, 
1988/2013). Jóhannesson (2006, p. 105) described the changes as based 
on "child-centred, humanistic, and egalitarian views ... apparent in 
cooperative learning methods, integration of subject matter, evaluation as 
a process rather than a product, and many other ‘progressive’ views in 
education." Many of these progressive ideas and views that articulated 
shared notions of/preferences for certain ideas and values are still discur- 
sive themes in the current Act of 2008 (Lég um grunnskóla, 2008), despite 
its neoliberal orientation (Schriewer, 2003). This reform had a strong 
mandate and research-oriented focus within the Ministry itself (i. 
Skólarannsóknardeild) and involved interactive cooperation with teach- 
ers to develop new curriculum materials (Edelstein, 1988/2013; 
Halldórsdóttir et al., 2016). 

The last reform prior to the period under review in this paper was the 
Education Act 1995, followed by a new curriculum in 1999. The empha- 
sis then was on decentralization, neo-managerialism, and individualiza- 
tion in the diagnosis of learning disabilities (Jóhannesson, 2006). The 
period under review was initiated with a new act on compulsory schools 
and process for development of a new curriculum, which was led by Ms. 
Porgeróur K. Gunnarsdóttir, a member of the Independence Party. The 
only Minister who was not part of the Independence Party during 
2008-2017 was Ms. Katrin Jakobsdóttir from the Left-Green Movement. 
She issued the curriculum for all school levels. 

The Education Act from 2008 and the 2011/2013 curriculum guides 
are still in force, and we note only one substantial addition—the 
Education Act (2015)—to establish a Directorate of Education.! 
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After the financial collapse in 2008, Iceland’s two political parties on 
the left formed a coalition and became responsible for dealing with the 
aftermath. Table 6.1 gives an overview of these political turbulences after 
the financial collapse in the year 2008 and which political parties and 
ministers were responsible for laws, regulations, curriculum and other 
policy papers during the period under review in this chapter. The first 
part of the fundamental educational reform had already been issued in 
the 2008 Education Act of All School Levels, and drafts of the new cur- 
ricula were available on the Ministry of Education, Science and Culture 
website. The discourse in the legal documents was focused on the com- 
petitiveness and quality of the Icelandic education system with in-text 
citations to the European Commission (European Qualification 
Framework) and institutions like the OECD (Alþingi [Parliament], 
2007). It was shaped with individualistic, market-oriented, managerial, 
and technological ideas (Dýrfjörð & Magnúsdóttir, 2016; Jónsson, 2018; 
Sigurðardóttir et al., 2014). 

The bill for the Compulsory School Act emphasized a competence- 
based curriculum, continuity and flexibility between school levels, qual- 
ity through formative testing and evaluation asserted by the establishment 
of the Directorate of Education on market and parental influence, and 
increased autonomy of schools and municipalities (Parliament, 2007) 
which was almost exactly the same emphasis found in the Norwegian 
reform several years prior (Karseth & Sivesind, 2010). The emphasis on 
gender equity (Guóbjórnsdóttir, 2003), democracy, and social justice 
that had been a strong part of the Compulsory School Act since 1974 
(Halldórsdóttir et al., 2016; Olafur Pall Jónsson, 2014) was kept and 
further nuanced. These nuances focused on the emphasis on inclusion (i. 
skóli án adgreiningar) as a policy that responded to the Salamanca 
Statement that Iceland signed in 1994 and had already impacted the 
whole education system. Iceland only had around 196 of students in spe- 
cial schools or units (Marinósson & Bjarnason, 2014). 

A political shift occurred when Katrin Jakobsdottir assumed the posi- 
tion of Minister of Education, Science and Culture, substantially influ- 
enced by an ethics report on the aftermath of the banking crisis (Árnason 
et al., 2010), emphasizing the strong democratic and critical role of the 
education system and the leeway for grounding that was needed to rewrite 
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the drafts of the curriculum (Jónsson, 2018). She sought to revitalize core 
values for the entire educational system as a foundation for a democratic 
society by emphasizing six fundamental pillars for all school levels: (a) 
democracy and human rights, (b) literacy, (c) sustainability, (d) equality, 
(e) health and well-being, and (f) creativity. During an interview in 2018, 
Jakobsdóttir explained: 


When our government came to power in May 2009, the work on the cur- 
riculum was in its initial stages and the approach was traditional, i.e., to 
focus on the subjects taught and list the fields that should be covered on 
different levels of education. But, from the very beginning, my main ques- 
tion was how do we put into practice the ideas of democracy that are the 
focus of our legislation? ... My vision was that the school had to be a basic 
democratic institution, that we needed to describe in more detail what this 
involved. (Jónsson, 2018, p. 62) 


The exclusive access of the Independence Party to state governance of 
education is relevant to this discussion, as had been in control consis- 
tently for approximately 20 years until 2009 when Ms. Jakobsdóttir took 
office after the global economic downturn in the autumn of 2008. The 
Independence Party then took the helm again from 2013-2017, which is 
the period of our study. These changes illustrate that the education gov- 
ernance landscape in Iceland during the fundamental and incremental 
reforms was disrupted by political shifts (Table 6.1) and economic crisis. 


Icelandic Governance 


The financial meltdown of the Icelandic banks in 2008 created not only 
an economic crisis but also a democratic crisis that directed attention to 
the governing body of the nation. The governmental and political foun- 
dation of the community was questioned. 'Ihe parliament organized a 
special research commission to investigate the failure of the banks. One 
of the issues uncovered concerned the lack of regulation and government 
supervision. “The most important lessons to draw from these events are 
about weak social structures, political culture, and public institutions. It 
is the common responsibility of the Icelandic nation to work towards 
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Table 6.1 List of Ministers of Education, their political parties, and the policy doc- 
uments that formed the fundamental and incremental education reforms 


Minister/political 


Year party Documents 

2003-2009 Porgeróur Act of Compulsory Schools 91/2008 
K. Gunnarsdottir/ 
Independence 
Party 

2009-2013 Katrin Jakobsdóttir/ National Curriculum Guides (2011/2013) 
Left-Green 
Movement 

2013-2017 Illugi Gunnarsson/ GP2014. Review of Policies to Improve the 
Independence Effectiveness of Resource Use in Schools 
Party Country Background Report 


WP2014. Hvítbók um umbeetur í menntun 
[White Paper on Education Reform] 

Education Act (2015) on New Directorate of 
Evaluation and Curriculum Materials 

GP2015. Mat á framkvaemd stefnu um skóla 
án aðgreiningar [Evaluation of the 
Implementation of the Strategy of 
Inclusive Education] 

GP2017: Report on School for All (2017) 
commissioned by the European Agency 


2017 Kristjan Por WP2017: Policy agreement on updating the 
Jülíusson/ Report on School for All (2017) into a 
Independence white paper. 

Party 


Note: Documents in Italics are part of the bibliographic analysis 


strengthening them and constructing a well-functioning democratic soci- 
ety" (Árnason et al., 2010, para. 4). 

Governance operations in Iceland have been criticized for lacking pro- 
fessionalism and democratic practices (Árnason & Henrysson, 2018). 
According to research on government practices in 2010-2011, Iceland 
deviates from the other Nordic countries in preparing bills for legislative 
acts. Ihe premises on which policies are based are generally not as system- 
atic in Iceland as in the other Nordic countries, especially Norway, 
Sweden, and Finland. In Iceland the Minister is given more autonomy to 
decide on procedures (Kristinsson, 2013). Compared to other countries, 
the independence of the Icelandic Minister has been enormous and the 
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role of the position loosely defined (Kristinsson, 2009). In 2010 the 
Ministry of Education, Science and Culture issued a report commis- 
sioned by experts/scholars in administration and politics that proposed 
that the role of ministers be more clearly defined in legal frameworks. In 
the report, the authors concluded that a minister should "seek profes- 
sional analysis from the ministerial administration before making deci- 
sions" (p. 3). This requirement is now part of the Act on Ministries Office 
(Forsetisraduneytid, 2010). 

The period under review is from 2013-2017 or a few years after the 
country went through a thorough discussion, reports, and stricter regula- 
tions on this process. Since the law (Lög um Stjórnarráð Islands, 2011) 
was issued, three white papers? have been published, one of which focused 
on education and is under review in this chapter. No research has been 
conducted to examine the governance practice that guided the policy 
document formulation in the Ministry of Education, Science and 
Culture; thus, this study is the first to explore that by drawing on empiri- 
cal data. 


Methodology and Research Design 


This frame of research is attended to explore the use of evidence in the 
Icelandic nation's policy formulation, how the evidence was used in a 
specific time frame, to what end, and by whom (authors, publishers). 
References are the primary definition of evidence that are used to “pro- 
vide legitimacy" for policy development decisions (Steiner-Khamsi, 
Chap. 2 in this volume). 


Methods 


The following data from the three documents in focus in this study are 
explored within this analysis: the publishers/authoring organizations of 
the documents’ references and citations and their geographical classifica- 
tion (domestic/Icelandic, Nordic/regional, and international); the type of 
knowledge (e.g., academic, reports, and laws/regulations) used; and the 
co-citations, which are studied through a bibliographical network 
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analysis. For the selected documents, we look at in-degree centrality, that 
is, how many times a given reference is cited in these three papers, as a 
measure of its significance in the policy discourse. 

Differently from Finland and Norway, the WP and GP as concepts 
have not been directly used in Icelandic policy making. For the purpose 
of this analysis, the WP is understood as an official document that for- 
mally suggests how to revise or develop new legislation or direct policy 
issued for state/municipality policy actors to implement. 

A GP in this study constitutes a document that is commissioned for 
the Ministry to obtain relevant background information to justify amend- 
ments within legislation or create new directions in policy making. For 
this reason, a GP is often used as a resource/reference for development of 
a WP, including the reference list or formal citations. 

In our case, we have examples of many documents that are beyond the 
scope of this study, as they were commissioned for institutions other than 
the Ministry, were not used as references, and/or lack a reference list. 
Therefore, the content analysis is important in determining how the GPs 
or other background documents are embedded within the texts. For this 
purpose, we used Atlas.ti word-counting and thematic analysis. Our 
analysis is designed to answer the research question: What kind of knowl- 
edge is used, interpreted, and recontextualized in the documents under 
examination? 

In order to gain some insight into the policy development process 
within the Ministry of Education, Science and Culture, five interviews 
were conducted with Ministry officials in order to inquire about Nordic 
and other multinational cooperation and to inquire about the modus 
operandi that underpins policy formation to clarify some aspects of the 
construction of the three documents. The interviews were recorded and 
transcribed, and interview subjects were assured of their confidentiality 
and anonymity. Instead of names, the first five capital letters in the 
Icelandic alphabet (A, B, D, E, F) are used to represent the interviewees. 
Jónasson conducted the interviews in December 2019. Interviewees were 
asked to participate due to their involvement in the policy formation 
process at various times from 2013-2017. 
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Selection and Types of Source Documents 


The only document that was an obvious choice for the analysis was the 
first educational document issued with “white paper” in the title 
(WP2014). The other document discovered as a WP that was part of the 
follow-up incremental reform is the audit report published in 2017 by 
the Ministry of Education, Science and Culture and the European 
Agency, originally written as a GP. It gained status as a WP after educa- 
tional leaders in Iceland signed a declaration of cooperation based on the 
report (i. Samstarfsyfirlysing). 

In 2013, the Ministry of Education, Science and Culture decided to 
initiate work to examine how the general policy of school for all worked. 
The European Agency for Special Education was designated to conduct 
an external evaluation of the situation concerning inclusive education, 
the policy on the issue, and its implementation. This task was undertaken 
in 2016 based on a special agreement among three ministries—the 
Ministry of Education, Science and Culture; the Ministry of Health; and 
the Ministry of Social Welfare—along with the Association of 
Municipalities, the Teachers’ Union, The Union of Upper Secondary 
Rectors, and the leading parental organization (i. Heimili og skóli). The 
evaluation was based on input and consultation received domestically 
through surveys, focus group interviews with stakeholders and experts, 
and a background paper authorized by Ministry officials that was not 
published (according to an interviewee). Thus, the document was ini- 
tially classified as a GP. The report was presented and discussed at a meet- 
ing on March 2, 2017. The ministers and the representatives of these 
organizations that had agreed to this initiative in 2016 signed a declara- 
tion in which they agreed to follow through on the actions proposed in 
the report. The status of the document was then changed from GP to WP, 
as it was brought into the policy arena by the main stakeholders, stating 
that “they will cooperate to follow up on the results of the audit report” 
(Ministry of Education, Science and Culture, 2017). 

The only GP that fulfilled all requirements was Document ID 3 (see 
Table 6.2), which was written according to a structure from OECD, as 
the document’s aim was to prepare for an OECD review for resource 
allocation and use in Icelandic schools. The Icelandic government 
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Table 6.2 List of documents for analysis 


Short Doc 
name ID Formal references in English Language 
WP-2014 1 Ministry of Education, Science and Culture (2014a). Icelandic 
White Paper on education reform [Hvítbók: 
Umbeetur í menntun]. Reykjavik: Ministry of 
Education, Science and Culture. 
WP-2017 2 European Agency for Special Needs and Inclusive English 
Education (2017). Education for All in Iceland— 
External Audit of the Icelandic System for Inclusive 
Education. Odense, Denmark: European Agency 
for Special Needs and Inclusive Education and the 
Ministry of Education, Science and Culture. 
GP-2014 3 Ministry of Education, Science and Culture (2014b). English 
Review of Policies to Improve the Effectiveness of 
Resource Use in Schools: Country Background 
Report: Iceland. Reykjavík: Ministry of Education, 
Science and Culture. 
GP-2015 4 Ministry of Education, Science and Culture (2015). Icelandic 
Evaluation of the implementation of the strategy 
of inclusive education. Report of a workgroup 
[Mat á framkvaemd stefnu um skóla án 
aðgreiningar. Skýrsla starfshóps]. Reykjavik: The 
Ministry of Education, Science and Culture. 


undertook the writing of a background report, intended to provide 
OECD with basic information, as is customary, which would be used as 
input for OECDS evaluation and recommendations. In the current con- 
text, it is important to keep in mind that its aim was to assemble infor- 
mation and provide an understanding of the system in preparation for an 
OECD evaluation. 'Ihe evaluation never took place, but the report was 
referred to in several subsequent documents, inter alia the WP2017 ana- 
lyzed here. Ihe document, which contains an extensive description of the 
Icelandic system, was prepared by the Ministry of Education, Science 
and Culture for OECD and the EC (European Commission), and the 
committee that oversaw its writing was represented by important stake- 
holders? Even though the focus of the report was on the three school 
levels, the expert group placed emphasis on the compulsory level. Two 
local experts assisted in drafting the document. Document ID 4 (see 
Table 6.2) is understood as a source document. It is missing a reference 
list, so it is not part of the analysis. 
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Results 


The documents analyzed came from fundamentally different directions, 
even though they were issued within a short time frame under the same 
Minister, which makes them particularly interesting for our analysis. The 
interviews are used to clarify the nature of the documents and the differ- 
ences in the reference profiles obtained. 


The Knowledge Base in the Documents 


This section presents an overview of the findings from the bibliometric 
analysis. The analysis is grounded in the location of references, types of 
documents, the network of references, and the publishers. 

A total of 203 references were included in these three documents, and 
more than half are referenced in the only GP that is mainly based on 
domestic publishers (Statistics Iceland and the state). Both the WPs con- 
tain relatively few references, and their reference lists are mostly based on 
international publishers in the form of reports, but similar to the GP, they 
include very few academic resources (Table 6.3). 

There is little overlap in the resources cited. The network structure 
(Fig. 6.1) shows how loosely related these documents are in terms of ref- 
erences, especially the two WPs: the only connection between them is 
that WP2017 refers to WP2014. Moreover, GP2014 is a reference in 
WP2017 and is mentioned as a work in progress in WP2014 without a 
formal reference. 


Table 6.3 Reference distribution in the source documents 


Documents Location of publisher Type of document 

Journal 
Source Initials Icelandic Regional Int'l Report Book article Gov't Others 
1 WP2014 26% 3% 71% 62% 9% 6% 20% 3% 
2 WP2017 29% 13% 58% 50% 0% 17% 21% 12% 
3 GP2014 95% 0% 5% 11% 3% 3% 71% 12% 
4 GP2015 0% 0% 0% 0% 090 096 096 0% 


Note: WP = White Paper; GP = Green Paper 
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= 


Fig. 6.1 The network structure of the three documents. (Notes: Regional = gray; 
domestic = white; international = black; node size = in-degree centrality. 
Source: circle) 


Figure 6.1 shows that nine documents are referenced in more than one 
of the source documents. Seven are governmental documents, and two of 
these documents are reports on the PISA results from 2012 (OECD, 
2013; Halldórsson et al., 2013). Accordingly, a joint focus in the two 
WPs concerns the Icelandic legal framework and PISA results from 2012. 


Emphasis on Quantitative Evidence from Reports 


In WP2014, the authors claim that the use of evidence is at the core of 
the proposed reform: “Work on the White Paper has been based on inter- 
national studies of education reform, and attempts have been made to 
draw lessons from the experience of those nations that perform strongest 
in international comparisons” (p. 5). 

The policy discourse in the WP2014 is based on information about 
those countries that outperform other nations in the comparisons and 
the aim is to learn from their reforms. The focus on best international 
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performance indicates that “best practice" guides policy development. 
Table 6.4 provides an overview of the figures that concern the compul- 
sory education part of the WP2014 indicating sources and actual refer- 
ences used. Three different perspectives are used in exploring large data 
sets.“ One is the international comparison that examines the positioning 
of Iceland within the large list of participating countries (Table 6.4: 
Figures 1, 2, and 3). The second perspective is the Nordic comparison 
(Table 6.4: Figures 8 and 18), and the third comprises the national data 
(Table 6.4: Figures 5, 7, 9, and 10), where both longitudinal and area- 
based comparisons are made. 

The policy discourse in the WP2014 is based on a technical report 
(UNESCO & IEP, 2012) published by the United Nations Educational, 
Scientific and Cultural Organization (UNESCO) and International 
Institute for Educational Planning (HEP). Overall, instead of referring to 
a list of GPs, the list of references is based on the following: (a) interna- 
tional think tanks, such as McKinsey; (b) information about the educa- 
tion system based on comparative data collected by European institutions 
like Eurydice and the European Commission; (c) conceptualization, rec- 
ommendations, and survey results from intergovernmental organizations 
like OECD; and (d) policy borrowing from other states/nations like 
Ontario in Canada. 

Therefore, it comes as no surprise that 71% of the references in 
WP2014 are international, and 62% are reports. The reference list con- 
tains 36 citations when including in-text citations, of which 12 origi- 
nated from OECD, which is the most cited publisher. This is not the case 
for the other two documents, as in the Icelandic database, OECD is the 
fourth most cited. Further analysis of OECD's impact can be found in 
Ydesen et al. (Chap. 11 in this volume). 

It is very clear from the format of WP2014 that the underlying com- 
parative evidence is quantitative data presented in the figures presented in 
the paper, which are described in Table 6.4. There are 12 figures in 
WP2014 and 11 of them are of international sources mainly from OECD 
with PISA in the forefront. In 7 figures the focus is on literacy. All of 
them originate from the PISA database except one that is based on data 
from Eurydice 
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With reference to the quantitative evidence, no figures or tables are 
presented in WP2017 nor in the background report prepared by the 
Icelandic working group. 

The words “teachers,” "students," "reading," and “reform” are fre- 
quently mentioned in WP2014, while "inclusive," "stakeholders," “sys- 
tem," and “support” are among the most common in WP2017. The 
approaches used in these two policy endeavors are strikingly different. 
This is, however, in line with the Icelandic tradition of having a wide 
scope of possibilities to form a policy document, which is further dis- 
cussed in the next section. 


Emphasis on “Evidence-Based” Practice: Focus on Teachers 


The WPs are quite different in terms of aims and openings. In WP2014 
the implicit assumption is that international data, like PISA results, 
address what is most important in education, and by implication, the top 
priority is to address this issue; in WP2017 it is in the hands of the 
national Ministry of Education, Science and Culture to define the main 
problems and themes to focus on to get Icelandic schools to become 
more inclusive. In WP2014 the paper sets out two measurable aims—the 
one related to literacy in the compulsory schools is designed to increase 
minimum reading PISA standards. Three main categories of action are 
included. One is to allot more time to Icelandic as a subject, another is to 
develop standards by measuring reading proficiency at various levels as 
each student progresses toward the end of compulsory school, and the 
third is to form the work of the teachers in more detail. 


An extensive analysis carried out by McKinsey (2007) concluded that the 
two most important factors influencing the performance of an education 
system were the education and the work of teachers. The same conclusion 
has been drawn in recent academic writings (Hargreaves and Fullan, 
2012). ... Recently, an expert panel on the continuing education and 
professional development of teachers was set up. The panel’s tasks will be to 
propose ideas and priorities regarding the training and support that teach- 
ers should receive in their work, and develop new teaching methods. The 
Ministry of Education, Science, and Culture has also started consultations 
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with teacher education institutions on the content of teacher education 
and increased cooperation in this field. (WP2014, pp. 38-39) 


This focus is well known in international research: controlling by num- 
bers through yearly evaluations where teachers are made responsible for 
achieving better results by adopting the "right" teaching practices (Brian, 
2009; Robertson, 2016). On the other hand, WP2017, which also 
emphasized teaching practices, took a different approach intended more 
so to empower teachers themselves to professionalize in an inclusive way, 
as illustrated by the following excerpts: 


The need to train teachers on using diverse teaching methods—especially 
within upper-secondary education—is highlighted. (p. 110) 


There must be flexible professional development opportunities for teacher 
educators to support their attitudes, knowledge, skills and ability to model 
inclusive teaching practice in their work. (p. 130) 


Teams of teachers and support professionals should work together to 
develop flexible frameworks for curriculum and assessment, together with 
teaching approaches that engage all learners and support their active 
engagement and participation in learning. (p. 135) 


The discourse in WP2017 is not based on “best practices” of teachers but 
to enhance practices to be in line with the inclusive ideology. 

In WP2014, the negative consequences of the constructivist ideals 
among Icelandic teachers were one of the assumptions made for the 
declining reading comprehension. A graph from the Teachers and 
Learning International Survey (TALIS) survey showed how teachers in 
Iceland believed more heavily in the constructivist model than in direct 
transmission, compared to teachers in some other nations (WP2014, 
p. 40). The claim was also made by referring (only through in-text cita- 
tion) to one small-scale research study by Savola (2010) comparing 
Finnish and Icelandic teaching practices in mathematics, emphasizing 
the good performance of the Finnish school system that relies more on 
direct transmission. When systematically observing teaching practices in 
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large-scale local research on compulsory schools (Óskarsdóttir, 2014), 
the results indicate that teachers in Iceland do still heavily rely on direct 
transmission. 

To promote WP2014, the Ministry of Education, Science and Culture 
held a conference on literacy to which all teachers in the country were 
invited? The publication of WP2014 and this conference were the first 
steps in what was called a “National Literacy Concord" (i. Pjédarsattmali 
um læsi) with its own logo and a song. A few weeks later, a critique on 
Beginning Literacy (i. Byrjendalzsi)° was issued in the form of a memo 
from the Directorate of Education.’ Beginning Literacy highlights an 
interactive approach, collaborative work, and the active participation of 
pupils as well as integrated language (Eggertsdóttir, 2007). The argument 
was that the schools that had adopted the Beginning Literacy approach 
were, on average, scoring lower than other schools on standardized tests 
(the 4th grade test) and that it was not an evidence-based practice. The 
Minister made a clear statement about this: “From now on, if there is an 
idea to utilize a particular practice or program it must be based on avail- 
able evidence that documents its effectiveness" (Skaptadóttir, 2015, p. 10) 

‘There was a harsh debate about this in the media, and the differences 
in the test scores were contested by scholars; for example, a sociology 
professor at the University of Akureyri showed how the difference in 
scores had been exaggerated visually (Póroddur Bjarnason, 2015). In the 
same week the founder of Beginning Literacy stated during an interview 
that it should not be the role of the Minister of Education, Science and 
Culture to choose the teaching methods used in the classroom (Arnarsson, 
2015). Thus, the PISA "shock" was promulgated in the media, and the 
new institution ("Act of Law on Directorate of Education," 91/2015) 
took the first action to deconstruct Beginning Literacy as an accountable 
reading method in terms of evidence and achievement. 'Ihe authors of 
this chapter are not taking a position on this debate. Rather, this discus- 
sion is intended to serve as a follow-up on how the main objective in the 
WP—to increase achievement in reading—was strongly related to a par- 
ticular best practice used by teachers and how the mediatization of some 
sort of a PISA shock was an essential factor in promoting it. 
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The data analysis indicated that the use of academic references in policy 
making is rather low (both in terms of books and academic journals); 
indeed, few local (Icelandic) or regional (Nordic) academic references 
were found. In WP2014 two in-text citations were based on a publica- 
tion in academic journals. These two are Fullan (2013) that is a commen- 
tary paper, and the other is a small-scale dissertation study comparing 
Finnish and Icelandic teaching practices in mathematics (Savola, 2010). 
An examination of the academic references in the database, published 
either as books or as articles in academic journals, revealed that none of 
those references includes the keywords “literacy/reading” or “inclusion,” 
which are the main educational themes in the two WPs. Instead, the use 
of reports to define and discuss the main concepts—literacy and inclu- 
sion—is dominant in both WPs. For the reading proficiency discussion, 
PISA is the main source. The use of academic papers was also scarce when 
arguing for changing direction in teaching practices. 

The reason for the limited number of academic references, both from 
international and national perspectives, was discussed during the inter- 
views. It seems that the research culture in Icelandic academia does not 
enter into the governmental framework due to the lack of large-scale, 
quantitative, and comparable research conducted in the academic con- 
text of Iceland. 


I think we need to use more systematically collected data. ... I think if the 
intention is really to influence policy, the systematic analysis based on large 
scale data sets is needed, but also from large scale [national] studies. But it 
requires an effort to do this. For example, the research from Geróur (refer- 
ring to Óskarsdóttir, 2014) has a lot of information. Still, it is, however, 
what you can find in it and pull out of it rather than systematically con- 
cluded results with statistical data that you can straightforwardly use in 
public policy making. (Interviewee B, December 2019) 


The Ministry official is asking for a different approach that is more con- 
venient for policy makers, that is, research that is more comparable and 
quantifiable (statistical) and easy-to-use results, reducing what then 
counts as evidence. The authors of the audit report (WP2017) did not 
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speak or read Icelandic and relied, instead, on international reports and 
their own data collection and analysis. 

It has been suggested that the policy field is gradually gaining more 
influence on the academic world, controlling what can count as valuable 
knowledge. In recent years the neoliberal discourse on market solutions, 
evidence-based research, and performativity have become more influen- 
tial (Ball & Olmedo, 2012; Dovemark et al., 2018). This orientation has 
challenged what had been the dominant model. In Iceland, stakeholders 
have most often been included in policy formation, such as with laws and 
curriculum, which is the case in two of the three documents under review 
in this paper: WP2017 and GP2014. Icelandic academia, however, has 
not been one of the stakeholders to have a right to be heard in the demo- 
cratic process; instead, research is produced by scholars independent of 
the field of policy, and therefore, it has been easy to bypass and overlook. 


The Icelandic Procedure of Education Policy 
Formulation: Ministerial Governance 


From the spectrum of available documentation, from the content of the 
documents analyzed, and from the responses of the interviewees, we can 
conclude that no established procedure exists to direct the construction 
of educational policy documents. According to Interviewee B, “There is 
no organization or a planned process that covers all policy making. ... 
There is no systematic analysis in the background.” Interviewee D 
explained, “To write a green paper with all the documentation and refer- 
ences, we sometimes skip this and enter the proposals for action stage.” 
As an example, the European Agency authorized the audit report on 
inclusive education (WP2017) because a long-standing Ministry official 
had for years led the work with European Association for International 
Education (EAIE) for Iceland. The official used their connection to con- 
tract with the Agency to prepare the report. Ihe EAIE asked for a back- 
ground paper, "Ihe Critical Reflection Paper" which became the third 
background paper issued by the Ministry of Education, Science and 
Culture in relation to policy formation. WP2017 was the fourth and final 
paper in this process. The lesson learned from the interviews is that each 
Minister determines how to proceed, often by designating someone who 
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is trusted to chair the work, either inside or outside the Ministry of 
Education, Science and Culture, who, in turn, may have considerable 
independent views on how to proceed. Indeed, no formal procedures 
apply. Interviewee A described it this way: "Talking about influence, the 
ministers views are much more influential than something that comes 
from outside." According to Interviewee E, "Substantial consultation 
may or may not take place; the evidence is usually collected, but not in a 
very systematic way, and the work may thoroughly integrate the ministry 
specialists, or not." 

The policy document formation process has not been institutionalized 
along certain operational lines, even though considerable expertise is 
often harnessed into the process. Furthermore, there is neither a tradition 
to formalize the evidence used (e.g., by systematic referencing) nor con- 
sultation obtained. 


“There Is Often a Feeling of an Urgent Need for Action": 
The Tight Timeframe 


As mentioned previously, the concepts of the WP and GP have not directly 
been in use in Icelandic policy making until very recently, which allows 
extraordinary leeway for a short-cut and powerful agendas to push for- 
ward in a short amount of time. “There is often a feeling of an urgent need 
for action. For example, if PISA results are bad, there may be little room 
for delay" (Interviewee D). 

Part of the core curriculum for all school levels was published in the 
beginning of 2013, and the new Minister of Education, Science and 
Culture, Illugi Gunnarsson, who took the position in the spring of 2013, 
managed to publish a WP in June 2014. It would have been impossible 
to issue a WP with the standard protocol in such a short time. Two of the 
officials in the Ministry of Education, Science and Culture mentioned 
that the main reason for the focus on PISA in WP2014 was a sort of PISA 
shock: “The white paper had mainly focused on OECD material. Other 
data somehow didn't make it into the paper. I think there was some kind 
of a PISA shock reaction that pushed this to the fore" (Interviewee D). 

This is exactly what has been defined as re-articulation of social justice 
to test-driven data and numbers (Lingard et al., 2014), where evidence 
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becomes the vehicle for reform and renewal in the education sector. The 
follow-up of WP2014 was massively resourced financially (compared to 
other projects) according to a Ministry official, and the Minister of 
Education, Science and Culture himself visited every compulsory school 
in the country to introduce it to the school communities. WP2014 was 
never passed through any kind of regulation in the parliament. The push 
for writing the first WP at the Ministry of Education, Science and Culture 
in such a short time frame, just a year after the subject-based curriculum 
was issued, could be understood as creating leeway for the Minister of 
Education, Science and Culture to bypass the new curriculum, change 
direction, and make his own agenda without the need for any influence 
from the parliament or other governance bodies. 

The audit report (WP2017) also had a tight time frame. The two back- 
ground papers were issued in 2015 (Mat á framkvemd stefnu um skóla án 
adgreiningar: Skýrsla starfshéps, 2015; Ólafsdóttir et al., 2015) and the 
cooperation with the European Agency started formerly in late 2015. The 
"Critical Reflection Paper" background paper, according to an inter- 
viewee, was written before data collection process, which started in spring 
2016. An enormous amount of data was collected in four days by the six 
audit members, as they spent just few days in the country. This included 
27 focus groups involving 222 participants, 11 school visits, and 9 indi- 
vidual face-to-face interviews with high-level decision-makers. An online 
survey was available for six weeks from May-June 2016 (WP2017, 
pp. 25-27), and the report was launched in Reykjavík on March 2, 
2017, the same day it was updated as a WP. 


The Small Nordic State in a Globalized World 


After analyzing our interview data, two discursive themes emerged from 
the reasons given for relying so extensively on external authorities. They 
concern the scarcity of officials to participate at the policy-making level 
and to interpret or translate the OECD discourse and data and the 
importance of externalization to avoid nepotism or as a strategy to change 
direction from what is suggested nationally. 
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Icelandic governance suffers from the lack of capacities at the Ministry 
of Education, Science and Culture, including the lack of domestic experts 
and human capital who can participate at the policy level and then filter 
the OECD influence and control it in line with the normative way of 
governance (e.g., through GPs). 


It is a major task to be on top of everything within the OECD, and the 
countries have different capacities available in terms of money and staff. 
Some, like Finland and Estonia, as examples, have people stationed in Paris 
who only attend to these tasks. And [some] countries don't have education 
attachés who follow what is happening and prepare meetings ... it is only 
a fraction of my job here to go there and attend meetings ... but this is 
supportive of our work, and we do our best. I have just been talking to a 
Norwegian colleague who deals with OECD, Nordic issues and bilateral 
issues—but he has a support of twenty staff. Yes, one feels a bit over- 
whelmed. (Interviewee B) 


The small state lacks human capital to cover all international cooperative 
endeavors and translate transnational effects and policy borrowing into a 
more local frame of policy discourse. In Finland a large group is desig- 
nated to do that and is authorized as such (Volmari et al., Chap. 5 in this 
volume). 

" [he eye of the guest is keenest" (i. glóggt er gests augaó) is a maxim in 
Iceland that captures the belief that we need to get somebody that is not 
part of “our group” to explore objectively and tell us the "truth" of what 
is really happening. The words from the Minister of Education, Science 
and Culture that are discussed in WP2017 reflect on this belief: 


At the audit launch event on November 3, 2015, Mr Illugi Gunnarsson, 
the Minister for Education, Science and Culture at that time, stated that 
the main motivation for the Icelandic stakeholders in requesting an audit 
was to gain an external view of the operation of the Icelandic system for 
inclusive education. He quoted an Icelandic maxim that says "Ihe eye of 
the guest is keenest.” (WP2017, p. 12) 


This viewpoint is based on the belief that Icelanders, in most cases, have 
dependent and biased viewpoints. For policy formation on inclusive 
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education, two background papers (Mat á framkvemd stefnu um skóla án 
adgreiningar: Skýrsla starfshóps, 2015; Steingerður Ólafsdóttir et al., 2015) 
were requested by the Ministry/Katrín Jakobsdóttir. 


‘There was ongoing work on analyzing the stage of inclusive education here 
in Iceland, which was ... not getting us anywhere. Then there popped up a 
suggestion of requesting help from the European Agency, just to get an 
external view, get the guests eye, and that was the end result. ... But a 
request was sent to them, as their approach is more democratic than the 
OECD’s and not with a tight structure and standards of indicators and 
questions as OECD. (Interviewee B) 


Sensibly, it was not a matter of whether to ask for external help—the 
choice was made between two international knowledge providers, OECD 
and EAIE, and EAIE was recommended to the Minister of Education, 
Science and Culture due to its democratic and inclusive governance prac- 
tice (according to one interviewee). One can also interpret this as a way 
to get a different political frame of reference than the national context 
could offer at that time. Externalization can also be a way of bypassing 
some local objectives, people, and beliefs. 


Conclusion and Discussion 


The external steering and input into policy making were substantial in 
Iceland from 2013-2017. GP2014 was written by Icelandic experts in 
line with a structure from OECD, and WP2017 was written and pub- 
lished by an external audit (European Agency) but processed by informa- 
tion from and in consultation with Ministry of Education, Science and 
Culture officials, Icelandic experts, and stakeholders. WP2017 is an 
example of a document where the externalization is achieved by getting 
external agencies to bring some possible solutions to the table. English is 
the original language of two of the three documents explored in this 
study. WP2014 is exemplary for its dominant use of international reports, 
especially from OECD, which is the most cited publisher, in defining 
main concepts, grounding arguments for the weak points in the Icelandic 
system (teacher practice), and providing solutions. 


174 B. R. Magnúsdóttir and J. T. Jonasson 


The use of references varies widely, both in scope and extent, with 
GP2014 accounting for 70% of the references used in total. In the com- 
bined database the primary knowledge providers are Statistics Iceland; 
Icelandic Ministry of Education, Science and Culture; the Icelandic 
Parliament; and OECD. According to the bibliographic analysis, aca- 
demic papers in general, but especially Icelandic papers, are thought to 
be irrelevant or not providing "accessible" knowledge in the evidence 
base for the policy. Strong transnational influences can be observed, and 
in WP2014 an excessive policy borrowing in terms of references and 
values, especially from PISA, OECD, Canada, and international think 
tanks, is evident. According to the interviews, the procedure in Icelandic 
education policy making is loosely structured compared to governmen- 
tal practices in countries where the procedure is more formalized 
(Kristinsson, 2013). 

The main focus was on the only WP that has been issued as such in 
Iceland, and analysis of the other two had the aim of providing a wider 
picture of policy formation. The over-emphasis on international and 
quantitative evidence in WP2014 was not as prominent in the other two 
papers. A more ethnographic approach was detected for the knowledge 
producing in relation to WP2017 with their own school visits and inter- 
views as main sources of data. In GP2014 the main source was Icelandic 
governmental data. WP2014 is exemplary when it comes to a scarcity of 
academic references and time frame. WP2017 did have a short time 
frame but was better articulated in the three background documents that 
had been written before getting the European Agency to write the paper. 
On the other hand, PISA-shock was the driving force for WP2014. The 
urgent need to react immediately to declining results in PISA and media 
attention can support an argument for ministers for policy changes (Sellar 
& Lingard, 2013). 

Prior research on the logic of practice in Icelandic governance reso- 
nates with our results in terms of loosely defined protocols and ministe- 
rial governance (Kristinsson, 2009, 2013). This also holds true in the 
Ministry of Education, Science and Culture. Despite the thorough dis- 
cussion, reports, and stricter regulations on the procedures in the gover- 
nance after the financial crash in 2008 (Árnason & Henrysson, 2018) 
reflected in the new law on governance (Icelandic parliament, 2011) the 
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different formulation with the two WPs reveals how loosely defined and 
ministerial governed this process still is. Ihe short time frame for publish- 
ing a WP in 2014, focusing solely on reading performance even though 
the Ministry of Education, Science and Culture had just recently issued 
a curriculum (2011/2013) that had not been implemented, is noticeable. 
On the other hand, this focus on literacy has existed for centuries in 
Iceland (Guttormsson, 2008). 

‘The scarcity of people working in international relations for the state 
education government and the distance between the policy and academia 
in the field of education in Iceland provide more space for the external 
voice to design national policy. It is tempting to rely heavily on interna- 
tional sources, probably to a greater degree than others, when the state 
governance lacks the human resources required to tackle all their transna- 
tional duties and to translate the transnational effects to an Icelandic con- 
text (Ydesen et al., Chap. 11 in this volume). This is also a way to avoid 
being accused of nepotism in this small country. However, in a ministe- 
rial governance ministers can bypass objectives and ideas that already 
have gained status in the form of regulations and previous and current 
discourse, for example, in the local academic arena, and use this danger 
of nepotism to gain leeway from that and find an international knowl- 
edge provider abroad that is more in line with their own political agenda. 

WP2014, the only document that was authorized and published exclu- 
sively by the Ministry, is a clear example of substantial externalization. It 
reveals the power of OECD and marketization of knowledge through 
rankings to produce, extract, and choose from the available knowledge to 
form a policy document. Transnational agencies manage to boil complex 
issues down to relatively simple numbers, to present sociological issues in 
terms of quantification. They come across as politically neutral in the 
service of local policy making. The data is seen as comparable and essen- 
tially problem free. So even if the evidence used sometimes only refers to 
the Nordic countries or even to one country, Iceland in our case, the 
reference base is still defined by OECD. Numbers and statistics have for 
a long time been used in educational research and also in policy making 
to understand the distribution of capital, poverty, educational outcomes, 
and so on. Numbers have been part of the national state for centuries to 
gain overview and control. The development in Iceland as it is 
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represented in WP2014 shows the dominance of the international per- 
spective and how governing by numbers is a way to ensure social justice 
and, thus, depoliticize political decisions. It is a technology of governance 
that aims at easing out politics and values that are based on other views 
and concurrently advocate for more efficiency, accountability, and effec- 
tiveness. It is an attempt at steering from above, especially aimed at teach- 
ers in WP2014, by collecting data that translate life in schools and 
communities into a series of graphs, grids, league tables, and indices and 
by introducing examples of best practices. This discourse is based on the 
view that competitive and test-driven education policy in the core sub- 
jects (reading and math) is the route to real social justice (Lingard et al., 
2014; Sellar & Lingard, 2013). 


Notes 


1. This was essentially a merger of two existing institutions concerned with 
evaluation and curriculum materials, with the addition of substantial 
administrative regulatory tasks that had been within the Ministry. It pro- 
vided important leverage for executing the international mandate on 
quality, competitiveness, and governing by numbers (Lingard et al., 2014). 

2. Three government papers, titled “white papers,” have been written in 
Iceland prior to 2020. The first was a white paper on environmental pro- 
tection (Gísladóttir et al., 2011), the second the white paper on educational 
reform in 2014, and the third is a white paper on a future vision for the 
financial system in 2018 (Blóndal et al., 2018). 

3. There were two representatives from the Teachers’ Union (one from the 
teachers in compulsory school and one from the principals) and two from 
the Icelandic Association of Local Authorities (p. 8). 

4, Here we are only looking at the literacy emphasis, which includes nine 
figures. 

5. The keynote was Dr. Maryanne Wolf, the author of the book: We Were 
Never Born to Read: The Story and Science of the Reading Brain. 

6. It is an Icelandic version of an interactive balanced approach. Half of the 
Icelandic schools have implemented it in 1st and 2nd grades. 

7. It was removed a few days after it was published due to harsh criticism. 

8. In October 2017 a translated version of the report was published by the 
Ministry of Education, Science and Culture. 
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Structuring School Reform Policy 
with Evidence: The Inter-mediational 
Role of Knowledge Sources 
and Arguments 


Bernadette Hormann and Kirsten Sivesind 


In Nordic countries, systematic inquiries have played a crucial role in the 
nation-states’ efforts to reform public education. However, in recent 
decades, various stakeholders have raised serious concerns about the legit- 
imacy of such inquiries. Both in media and in research, critiques have 
targeted the quality of professional knowledge, suggesting that education 
policy should draw on scientific evidence to reform and evaluate the edu- 
cation system. This chapter examines the institutional response to this 
critique by examining how national authorities have made policy into an 
evidence-based pursuit of ministries and their governmental bodies. 

By inquiring about two white and eight green papers published by the 
national authorities in Norway, we ask the following questions: How do 
policymakers and experts provide evidence and expertise in issuing school 
reforms for basic education? How do they identify options for school 
reform by deploying knowledge through argumentative forms of 
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reasoning? How do various types of evidence structure these options by 
connecting policy realms and systems? We have divided the chapter into 
three main parts. 

First, the chapter begins by establishing the policy context, the theo- 
retical background for our study, and a typology that classifies various 
knowledge sources we will use for analytical purposes. We present some 
contextual information about the school reform we examined and give an 
overview of the bibliographic meta-data that we collected and the research 
strategies we used to examine the mediation and use of policy-relevant 
knowledge. We also introduce perspectives from sociological system the- 
ory based on Luhmann (2018) and associated scholars (Andersen, 2019). 
This theory helps us examine how references to knowledge sources are 
semantically translated within policymaking processes and how evidence 
informs thematic areas in school reform policy, such as curriculum and 
assessment. In addition, the theory serves as a link between policy pro- 
cesses and policy systems, such as science, politics, and education. 

Second, we present the results obtained from our bibliometric network 
analysis in the form of the frequency and distribution of prominent refer- 
ences in our dataset. By looking into the ways two white papers and eight 
green papers refer to various knowledge sources (e.g., research reports, 
reviews, and governmental documents), we demonstrate how policymak- 
ers use bibliographic references to strengthen their arguments for reform- 
ing basic education in Norway. In our analysis, we identify how some 
references acquire a prominent role by being referenced by several sources 
within and across policy realms. We also uncover their prominence by 
mapping whether these or other knowledge sources are frequently and 
explicitly in-text referenced and thereby influential within translations 
made by the authors. 

Third, we present a semantical analysis of how state authorities trans- 
late knowledge sources in-text referenced within the two white papers. 
These documents prepared for political decision-courses within the 
Norwegian parliament (see Chap. 10 for further details about the pro- 
cess). We analyze how policymakers formulated options for school reform 
through postulations and aspirations about structures, processes, and 
outcomes. Thus, we agree with Colebatch and Hoppe' argument that 
“policy may be seen as both ex ante intention, ex durante becoming, and 
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ex post outcome" (Colebatch & Hoppe, 20182, p. 6). Moreover, accord- 
ing to Luhmann (1990), a medium, such as a policy document and its 
references, can provide meaning through forms of reasoning by semanti- 
cally linking structures, processes, and outcomes via input/output 
schemes. Through these schemes, policy formulation and translation pro- 
cesses become programmatic by character (Luhmann, 2018). 

Finally, we question whether the most cited and prominent knowledge 
has influenced the semantic repertoire of themes and arguments and 
thereby the configuration of the decision programs built into the national 
school reform. Based on a comparison of how references are used within 
and across two main thematic areas covered by the white papers, we con- 
clude that evidence has a structuring impact on the formation of policy 
realms and, as a result, options for school reforms. The empirical investi- 
gation shows that claims and recommendations within the white papers 
are shaped by the type of knowledge source policymakers use, which 
policy realms they reference, and how they translate knowledge through 
argumentative modes of reasoning. We also demonstrate how policy doc- 
uments mediate evidence that structurally connects the political system 
with the education system. This demonstration leads to a discussion of 
whether and how professional knowledge is restrained or constricted by 
evidence-based reform within the education sector. 


The Use of Evidence and Expertise Within 
Policymaking Processes 


During the last decade, interest in evidence-informed policy and practice 
has increased in Europe and beyond. Such policy often favors scientific 
methodologies and empirical research as a frame of reference. Importantly, 
the term evidence does not bear much significance in itself since it is asso- 
ciated with various actors, different types of knowledge sources, and mul- 
tiple forms of knowing (Boaz et al., 2019, p. 5). Moreover, any material 
base that carries knowledge or any source of information may provide 
evidence depending on context variables (Sedlacko, 2018). Therefore, 
what counts as evidence and how evidence is used in reform policies are 
highly contingent questions. 
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This chapter examines how evidence serves as a medium in policymak- 
ing among experts who were mandated to develop official policies aimed 
at reforming a national education system. We investigate in particular 
how policymakers and experts selected research-based knowledge along- 
side other types of documentation, and how the policy documents medi- 
ated this knowledge as evidence to inform argumentation and 
decision-making processes. As Colebatch and Hoppe (2018b) have 
noted, “Both the documentation and the widespread ‘consultation’ 
within administrations can be part of the signature of policy-evidence 
that ‘due process’ was followed" (p. 15). As such, evidence-based policy is 
not merely a question of what kind of knowledge and information are 
produced and selected; rather, it is a question of how policy processes 
facilitate the usage and translation of evidence that various actors call for. 

Because the term evidence has become a buzzword with no clear defini- 
tion, both policymakers and practitioners are juggling multiple forms of 
evidence within these processes and involving various groups of actors 
that provide support for defining the "best evidence." Within this per- 
spective, the process of evidence use is considered a highly pragmatic 
enterprise. Despite attempts to establish clear hierarchies of evidence, few 
policy realms reflect a master plan for how to make use of expertise in 
reform-making processes. For example, in the Oslo Institute for Research 
on the Impact of Science (OSIRIS) project, Thune (2019) examined the 
use of evidence within public administrations in Norway. Thune found 
that both policymakers and practitioners deployed a variety of methods 
to access information and knowledge, such as contacting colleagues and 
conducting web searches. Moreover, policymakers used different types of 
media to collect knowledge sources, such as newspapers, publications, 
presentations, and informal dialogues. Against this background, one 
might wonder if there are any patterns that actually structure evidence 
use in policymaking processes. 

Despite the micro-politics of providing evidence, researchers have 
argued for the importance of observing patterns or forms of patterning of 
policy processes. From this point of view, policy researchers consider the 
character and strength of the ties of actors as one possible structure that 
characterizes the use and translation. of information (Honig & 
Venkateswaran, 2012). Among several other researchers, Thune (2019) 
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found that provision of evidence is dependent on the capabilities of 
actors. According to Thune’s survey results, persons who are in senior 
positions, are well educated, and have work experience from other sec- 
tors, especially research sectors, are more capable of collecting and using 
knowledge sources than persons in junior positions. Moreover, the influ- 
ence of these groups of actors is dependent on the relational ties that 
connect these actors. 

O’Day’s (2002) study of accountability reform in Chicago during the 
early 2000s serves as another excellent example of how such ties evolve. 
O’Day investigated how policymakers and professionals interacted with 
various forms of governance through the reception and translation of 
information. In an empirical mixed-methods study, she uncovered criti- 
cal mechanisms that enforced and constricted the flow of information 
within the policy-praxis nexus. A key assumption underlying this and 
similar studies is the changing role of bureaucratic governance. When 
studying the emergent impact of accountability systems through ties 
between administrative levels, O'Day (2002) identified impacts of both 
an outcome-based bureaucratic mode of governance and a professional 
mode of governance. Different from traditional bureaucratic govern- 
ment, these new modes centered on practice-based knowledge, 
performance-based standards, and bureaucratic accountability. 

In our chapter, we look at similar patterns, but primarily by focusing 
on documents considered powerful media for policymakers to create 
options for reform and change. We examine how policy documents 
mediate knowledge and information through their selection of references 
and deploy argumentative modes of reasoning that connect politics, poli- 
cies, and practices. Similar to O'Day (2002), we refer in particular to two 
models for how policies bring about change by referencing and translat- 
ing knowledge across policy realms and levels: a traditional bureaucratic 
mode for policymaking that aligns broad outcomes with general man- 
dates and an outcome-based bureaucratic mode that pursues means-end 
reasoning along with performance standards. Both these modes operate 
within the same context and involve different degrees of normativity and 
prescriptive routes of actions, and they can also be combined into mixed 
forms, as demonstrated in recent research on policy borrowing and lend- 
ing (Sivesind et al., 2016). 
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In our study, we draw on Luhmanns (2018) distinction between two 
program forms to map these alternatives. According to this theory, a con- 
ditional program and a purposive program differ by representing two sets 
of conceptual schemes for observing policymaking processes. According 
to Luhmann (2018, p. 213), a key difference between the two program 
forms is that they reflect various distinctions for observing decisions. 
While conditional programs divide between conditions and consequences 
to build up an argument about change, purposive programs distinguish 
between means and ends to observe problem-solving processes. This dif- 
ference leads in the next step to various modes of reasoning, based on 
expectations that can be more or less normative and future-oriented and 
that can be more or less comprehensive or narrow by ways of issuing a 
reform within the education sector. 

Moreover, by looking at policy documents as media for observing deci- 
sions about reform, we can also assess how evolving semantical structures 
are loose or fixed based on how they are formed. By examining how doc- 
uments are linked to other media (e.g., other documents) or to systems 
(e.g., science, education, and politics), we can assess to what degree poli- 
cymaking is structured (Andersen, 2019, p. 81). This examination may 
lead to an interesting discussion about the structuring role of evidence in 
policymaking processes if, for example, references to scientific evidence 
condition a reform by fixating particular modes of reasoning along with 
a purpose that national and international stakeholders have actually 
called for (Burns & Schuller, 2007). 

We analyze more or less fixed patterns of evidence use by comparing 
how documents are connected within and across two thematic areas or 
realms: curriculum reform and assessment practices. We demonstrate 
how policy documents for each realm and their references to knowledge 
sources connect reform and change both in bibliometric networks and 
through semantical patterns. We show how policymakers create decision 
programs for education that make up policy options that are more or less 
future-oriented and normative by their use of evidence and language 
(Luhmann, 2018, p. 2015). By looking into the way mediating links are 
configured between the source documents and referenced documents, by 
comparing the patterns across policy realms, and by assessing how they 
connect with science, politics, and education as surrounding systems, we 
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are able to analyze how evidence structures policy options for school 
reform in both a fixed and a loose form. Thus, we identify how the inter- 
mediational roles of both knowledge sources and arguments become 
decisive for the configuration of policy processes in the field of 
school reform. 


Reform, Data, and Methods 


In this study, we have analyzed the distribution and networks of biblio- 
metric reference in two white and eight green policy papers that were 
prepared for political processes within the Norwegian parliament on the 
most recent school reform in basic education (Years 1-13). They were all 
written under the auspices of the Norwegian government and the 
Ministry of Education and Research. The reference use and the argu- 
ments made for reforming and renewing the education system prepared 
for political decision-making processes within the parliament. Through 
the next steps, these documents resulted in the renewal of the Knowledge 
Promotion Reform (2016/2020) within the Norwegian education system 
(Baek et al., 2018; Steiner-Khamsi et al., 2020). The national govern- 
ment launched the reform in 2017, and it was formally put into effect in 
August 2020. This reform set out to renew the curricula for the core 
subjects in primary and secondary education and provided a new intro- 
ductory part that aimed at establishing a coherent framework for organiz- 
ing and assessing teaching and learning in schools (Sivesind & 
Karseth, 2019). 

To begin our analysis, we examined the bibliometric references in the 
two most prominent governmental reports (white papers, referenced here 
as WP£1 and WP#2) that were prepared for decision processes within the 
parliament. Thereafter, we included references in all public inquiry 
reports that were referenced in these white papers. Altogether, this sample 
made explicit references to 2312 knowledge sources, which were listed 
within the documents’ bibliographies and footnotes. By conducting a 
bibliometric network analysis, we identified the most co-cited knowledge 
sources within this corpus of documents. 
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For the bibliographic network analysis, our data consist of 2312 refer- 
ences from the reference lists of two governmental reports (white papers) 
and eight official reports (green papers, known as Norges offentlige utred- 
ninger [NOUS], written by experts who initially evaluated existing educa- 
tion systems and who addressed how to renew education in the future) 
referenced in the two white papers (Table 7.1).! 

This sampling reflects the logic of the reform-making process and 
allows us to analyze the "official knowledge" from which bureaucrats and 
politicians eventually draw. The references of our ten source documents 
were analyzed and edited with the software programs UCINET and 
Netdraw. 'Ihese programs generated descriptive statistics and visualized 
relationships between the documents. In this way, we identified the most 
prominent references of our dataset: those that were most often cited in 
the reference lists and played crucial roles in the reform discourse. We 
established a cut-off point at five, resulting in a list of 12 documents that 
were cited five to eight times in the whole dataset. Since we consider 
these publications essential knowledge in the process of reform formula- 
tion, we decided to track them back to the text in which they were actu- 
ally quoted. These documents are the main data for our semantical 
analysis. 

In addition to the bibliometric study, we conducted a semantical anal- 
ysis of how policymakers translated references through their argumenta- 
tive modes of reasoning. In this part of the study, we examined how 
policymakers formulated arguments to make recommendations for how 
to reform education in and across policy realms, such as curriculum 
reform and assessment practices. The school reform was formally built on 
two white papers (WP£1 and WP#2) written on behalf of the Norwegian 


Table 7.1 Sampling for the bibliographic network analysis? 


White paper #1: White paper #2 
Year 2015-2016 Year 2016-2017 
Topic: Curriculum renewal Topic: Quality monitoring 
Source documents Green papers 1-5 Green papers 5-8 
(N = 10) NOU 2003:16; NOU 2007:6; | NOU 2009:18; NOU 
White papers (n = 2) NOU 2014:7; NOU 2015:2; 2010:7; NOU 2015:2; 
Green papers (n = 8) NOU 2015:8 NOU 2016:14 


Total references 2312 
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government. These two white papers referred to various expert commis- 
sions and other sources that included references to articles, book chap- 
ters, and research reports, among others. Report No. 28 to the Parliament, 
Subjects, In-Depth Learning—Understanding. A Renewal of the Knowledge 
Promotion Reform (WP#1; Ministry of Education and Research of 
Norway, 2015), was published in 2015-2016 and presented a clear 
framework for curriculum revision. The report suggested continuing 
with and further developing competence descriptions, setting prioritiza- 
tion by defining core elements in the subjects, including basic skills in 
curricula, and creating more connections between the subjects by defin- 
ing three prioritized cross-disciplinary topics (i.e., democracy and citizen- 
ship, sustainable development, and health and mastery of life). In 
addition, the report suggested new goals and assignments for the forth- 
coming renewal process. Report No. 21 to the Parliament, Eager to 
Learn—Early Intervention and Quality in Schools (WP#2; Ministry of 
Education and Research of Norway, 2016), was published in 2016-2017 
and addressed diversity problems regarding students needing extra sup- 
port. WP#2 also presented a comprehensive framework to improve 
equity and quality with new models for monitoring education. 

For analytical and heuristic purposes, we identified two main groups 
of arguments: (a) those that maintained traditional governance modes 
that guide actions by regulating conditions and (b) those that purposively 
emphasized the need for reforming education more strategically. For this 
part of the analysis, we used Luhmann’s (2018) program forms (i.e., con- 
ditional versus purposive programs). Conditional programs are past- 
oriented and emphasize formal and substantial rationales for reforming 
education, while purposive programs are future-oriented and emphasize 
value-based and standard-based tools for achieving certain outcomes 
(Sivesind et al., 2016). For identifying how prominent references were 
used to legitimize arguments for school reforms and investigate the 
underlying communication patterns, we consider the combination of 
semantic and bibliometric network analysis to be a fruitful research 
approach (Froehlich, 2020). 

Finally, we conducted a structural comparison of reference use within 
and across the two white papers (Sivesind, 1999). This comparison aimed 
at developing insights into the third research question, addressing how 
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various types of evidence structure policy options by linking thematic 
areas and by connecting systems such as policy with education, politics, 
and science (Andersen, 2019). First, we examined if and how the most 
cited references shaped argumentative modes of reasoning through their 
tendency to favor a conditional program, a purposive program, or both. 
Through this comparison, we investigated if and how certain types of 
evidence can actually structure policy options by representing a particular 
type of knowledge. Second, we synthesized our findings and interpreta- 
tions about the argumentative translation of evidence within the white 
papers and asked how policymakers shape policy options through the 
expertise they deploy during the writing process. Moreover, we examined 
whether we found different patterns between the two policy realms in 
terms of how evidence was translated. Finally, we compared the ways in 
which the most prominent references in the two white papers connected 
policy realms by referencing the surrounding world of systems, such as 
education, science, and politics. 


Typology 


Along with Stephen Toulmin (1958), we consider a persuasive argument 
as consisting of both claims and assertions backed by evidence. These 
assertions refer to both facts in terms of data and information, and include 
reasoning that enhances a persuasive use of the argument. In addition, we 
categorize various types of knowledge sources from our dataset that were 
used to build an argument. Moreover, to compare the influence and 
usage of various types of knowledge as manifested within the policy 
papers, we have developed a typology that fits with our data that we can 
use to build arguments ourselves. This typology is based on existing clas- 
sifications of research and information that we combined for our particu- 
lar purpose. 

Doyle (2003) referred to Noblit and Hare (1988) when presenting 
three approaches to conducting research reviews: meta-analysis, literature 
review, and meta-ethnography. These approaches differ in their degree of 
incorporating contextual knowledge. Meta-analysis synthesizes numbers 
by aggregating findings to explain a phenomenon, such as what might 
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increase learning achievement. In this approach, researchers gather all 
available information from studies that have measured the same variables 
and explain key mechanisms via either the exhaustive collection of 
research articles or random sampling. On this basis, experts can present 
generalizations as a universal standard that can be applied (more or less) 
independent of contextual knowledge. 

The second alternative is the literature review, where the overall pur- 
pose is to create a chain of reasoning that helps illuminate the phenom- 
enon under study. In this case, both the theory and the results are 
considered relevant knowledge. Data collection must be exhaustive, and 
researchers are challenged to logically bridge summaries of results and 
interpretations for different studies. 

Within the third group, where reviews are based on meta-ethnography, 
the validity issue is particularly challenging. Doyle (2003) argued that the 
aim of generalizability in meta-ethnography is based on case studies 
where the local context shapes the ways in which findings and results are 
interpreted. Still, she asserted that it is possible to draw conclusions across 
case studies through synthetization, which generalizes knowledge beyond 
what is valid for single cases in the study. Such a generalization requires a 
particular methodology, which includes alternative analytical steps and 
aims at developing an increased understanding of a particular phenome- 
non. In many ways, this procedure is consistent with what Gough et al. 
(2012) labeled configurational synthesis, or the compilation of descrip- 
tions, interpretations, and results that are reorganized based on analytical 
steps and concepts rather than on pre-determined concepts or aggregated 
results. 

Beyond evidence-based reviews, knowledge sources can be scientific by 
being built on empirical data collected by researchers who follow known 
procedures and general standards to examine a particular research prob- 
lem. Such studies can be academic without any purpose of intervention, 
they can be clinical and result in knowledge to be applied for particular 
purposes, or they can be part of research projects and evaluations that 
guide actions through recommendations. According to Rasmussen et al. 
(2007), knowledge sources of this kind can serve different functions. 
Therefore, they suggested dividing educational research developed for sci- 
entific purposes from educational knowledge sources and theories used 
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for programmatic and practical purposes. Rasmussen et al. (2007) classi- 
fied evidence-based studies belonging to the first category as applied 
research that can be useful for policymakers and practitioners. A core aim 
in such applied research is to serve clinical purposes and project best prac- 
tices by the use of assessment standards. 

In our study, we include four types of reference sources that provide 
knowledge and information to support arguments within policy docu- 
ments: (a) formal documents that are not primarily a result of research 
but can indirectly mediate research and serve as one type of evidence, (b) 
meta-analysis, (c) configurational reviews, and (d) empirical research 
studies. These four types of knowledge sources do not cover all knowl- 
edge and information that could be used; for example, we have not 
included experience, practice-based innovations, or theoretical knowl- 
edge in our analysis. However, through our policy documentation, we 
can examine how these four types of knowledge sources are used and 
translated as well as how they serve as a structural condition for how 
policy options for school reform are expedited. 


Bibliometric Network Analysis 


An earlier examination of the 2020 reform revealed that the most cited 
knowledge sources within the bibliographies were predominantly of 
domestic origin and represented an interesting mixture of references, 
such as formal documents, research reports, and scientific publications 
(Baek et al., 2018). Moreover, the most cited references were research and 
policy reports that had not undergone a scientific peer-review procedure. 

Figure 7.1 shows the network structure of all references in our data- 
base. Source documents are visualized as circles and ordinary references as 
squares. The sizes of the nodes indicate their in-degree centrality, which 
is an indicator of the impact of a text assessed by its number of citations 
in other works. The shading shows the geographic origins of the refer- 
ences with regional nodes colored gray, Nordic references white, and 
international references black. 

The bigger nodes located in the middle visualize references cited by 
more than one report and how they are connected to each other. These 
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Fig. 7.1 Complete network structure of all references in our database. (Note: The 
network structure reveals that the publications are highly specialized and issue- 
centered, with little overlap between the various reports) 


co-cited references represent the core knowledge shared by one or more 
reports and are therefore the focus of our attention. It becomes apparent 
that there are a relatively small number of co-cited references, which 
means that the cited publications are highly specialized and issue- 
centered, with little overlap between the various reports (Baek et al., 
2018). For greater insight into the network of co-cited references, we 
made the following list of the 12 most cited references (i.e., cited more 
than five times) in the database (see Table 7.2). 

The document at the top of the list, the white paper Culture for 
Learning, is a governmental report to the Norwegian parliament from 
2003 to 2004. This foundational paper initiated the Knowledge 
Promotion Reform that was launched in 2006 and created systemic 
change in Norwegian education policy by introducing a national test sys- 
tem. Since policymakers presented the new renewal of the Knowledge 
Promotion Reform (2016/2020) as incremental and not foundational, 
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we find it reasonable that this document from 2003 is the most refer- 
enced paper. 

The second listed publication is Hatties Visible Learning (2009), a 
meta-analysis of studies on the effect of different educational influences 
on student performance. This book is not only omnipresent in media and 
research about direct instruction in schools, but it also is on the top of 
what is considered to provide evidence according to academic classifica- 
tions (Doyle, 2003). Number three on the list is a Danish report on 
teaching competence and student learning in kindergarten and school 
that synthesizes international research about the relationship between 
teacher competence and student learning in schools. Nordenbo et al. 
(2008) authored this report on behalf of the Norwegian Ministry of 
Education. Because the synthesis builds on existing research on student 
learning with a focus on "what works," it represents another typical 
example of evidence-based knowledge. 

The rest of the publications are predominantly governmental or official 
reports (NOUs), which we have categorized, together with the white 
paper Culture for Learning, as formal governmental reports. The two 
empirical research studies from Aasen et al. (2012) and Nordahl and 
Hausstátter (2009) contain results from evaluation projects on the 
Knowledge Promotion Reform from 2006. While Aasen et al. (2012) 
presented a comprehensive evaluation of reform and governance within 
the Knowledge Promotion Reform, Nordahl and Hausstatter (2009) 
analyzed a specific group of students, those in need of special support. 
‘The research article from Durlak et al. (2011) reviewed existing knowl- 
edge about the improvement of student performance by fostering social 
and emotional learning. The study by Durlak et al. (2011) was published 
in the journal Child Development, which has a comparatively high impact 
factor of 5.024, according to the publisher's website. The article referred 
to student performance and provided evidence-based knowledge about 
factors that can improve this performance. Along with Hattie’s Visible 
Learning, we classified the Durlak et al. (2011) article as a scientific pub- 
lication in a narrow sense. 

All in all, the list consists of publications that provide an interesting 
overview of prominent knowledge referenced by the school reform. This 
list reflects a mixture of formats (i.e., governmental and official reports, 
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academic publications, and policy reports) and origins (i.e., national 
reports, international research, and one regional/Nordic report). Moreover, 
the group of knowledge sources represented in Figure 7.1 can be analyzed 
and compared according to their levels of evidence. Drawing on Doyle 
(2003), Gough et al. (2012), and Rasmussen et al. (2007), we have dif- 
ferentiated between the following types of knowledge sources: meta-anal- 
yses, configurational reviews (i.e., theory-driven reviews of qualitative and 
quantitative studies), empirical research studies, and governmental papers. 
Table 7.2 shows the allocation of references to categories. 


The Argumentative Translation of References 


As a first step in our analysis of the argumentative translation of refer- 
ences, we identified knowledge sources that were most often referenced 
within the two white and eight green papers. In addition, we identified 
how often references appeared as in-text citations within the two white 
papers that are the basis of our semantical analysis. As shown in Table 7.2, 
the report on the evaluation of the Knowledge Promotion Reform (Aasen 
et al., 2012) had an overwhelming in-text citation quote of 20, while the 
white paper Culture for Learning from 2003-2004, which was the most 
co-cited document within the full database, was cited only six times. 
Moreover, John Hattie’s Visible Learning was co-cited seven times within 
the bibliographies of the ten texts and referenced eight times within the 
texts themselves. This pattern indicates that Hattie’s meta-analysis has 
been among the most influential references to project options for reform- 
ing schools in Norway. However, a comparison of the actual citations 
with those from the bibliometric network analysis makes it clear that the 
report by Aasen et al. (2012) was the most influential reference for the 
translation of evidence within the two white papers, as it was prominent 
on the overall lists of the ten documents and was the most cited source in 
the white papers themselves. 

To put the most cited references in perspective of our two white papers, 
we decided to compare the prominent sources within the full database 
with the total number of quotations of all references listed in the two 
white papers. The frequency of Aasen et al.’s (2012) evaluation report in 
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this analysis was topped only by NOU (2015, p. 8) The School of the 
Future. Renewal of Subjects and Competences, which was cited 35 times 
altogether. The rest of the references in the two white papers were cited 
eight times or fewer, which underlines the importance of the Aasen et al. 
(2012) report in the two governmental papers (see Table 7.3) alongside 
the green paper authored by the public inquiry commission about the 
school of the future (NOU, 2015, p. 8). 

In the list of the most referenced in-text citations (Table 7.3), we also 
found an expert report (Dahl et al., 2016) that was not a commissioned 


Table 7.3 Frequency of references (in-text) in the two white papers, more than 
eight citations 


No. of 
Title of reference citations 


NOU, 2015: 8 Fremtidens skole: fornyelse av fag og kompetanser 35 
[The School of the Future: Renewal of Subjects and Competences]. 
Ministry of Education and Research 


Aasen, P., Møller, J., Rye, E., Ottesen, E., Preitz, T., & Hertzberg, F. 20 
(2012). Kunnskapsløftet som styringsreform—et løft eller et løfte? 
Forvaltningsnivåenes og institusjonenes rolle i implementeringen 
av reformen [The Knowledge Promotion Curriculum as a 
Governance Reform—Promotion or Promise? The Roles of the 
Administrative Levels and Institutions in the Implementation of 
the Reform] (NIFU Rapport 20/2012) 


Dahl, T., Askling, B., Heggen, K., Kulbrandstad, L. I., Lauvdal, T., 8 
Qvortrup, L., Salvanes, K. G., Skagen, K., Skrøvset, S., Thue, F. W., & 
Mausethagen, S. (2016). Om lærerrollen. Et kunnskapsgrunnlag 
[On the Role of the Teacher. A Knowledge Basis]. Fagbokforlaget 

Dale, E. L., Engelsen, B. U., & Karseth, B. (2011). Kunnskapslgftets 8 
intensjoner, forutsetninger og operasjonaliseringer: En analyse av 
en laereplanreform [The Intentions, Preconditions, and 
Operationalizations of the Knowledge Promotion Reform: An 
Analysis of a Curriculum Reform]. University of Oslo 

Hattie, J. (2009). Visible Learning—A Synthesis of 800 Meta-Analyses 8 
Relating to Achievement. Routledge 

St. Meld. 28 (2015-2016). Fag- Fordypning—Forstáelse. En fornyelse 8 
av Kunnskapsløftet [Subjects—In-Depth Learning—Understanding. 

A Renewal of the Knowledge Promotion Reform] (White Paper No. 
28 to the Storting). Ministry of Education and Research 


Note: The two references highlighted in gray, Aasen et al. (2012) and Hattie 
(2009), were also among the most cited documents in the full database 
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paper, but still of crucial importance in argumentation for a new reform. 
In addition, we discovered another research evaluation report by Dale 
et al. (2011) that analyzed the national curriculum and its implementa- 
tion. The list also includes WP#1, which was referenced in WP#2. 

To trace the translations of these references and how they were shaped 
by argumentative modes of reasoning, we analyzed the paragraphs where 
the knowledge sources were referenced according to Luhmanns (2018) 
program forms, namely, the past-oriented conditional program versus the 
future-oriented purposive program. This analytical distinction helped us 
to assess if and how different types of reviews and reports were used to 
stabilize conditions that were already in place or to change education in a 
more strategic way. In this part of the analysis, we looked at the meaning 
units within the text, which consisted of one to three paragraphs follow- 
ing the reference. Moreover, we evaluated whether the argumentation 
manifested within the document referred to normative or descriptive 
statements. In so doing, we identified whether statements and arguments 
referred to past conditions and actions (e.g., decisions about what to 
teach in schools) or future activities and results (e.g., what to accomplish 
and achieve). This way of conducting semantic analysis helped to clarify 
how the documents symbolized certain program forms and a regulative 
or a strategic use of references. Based on this analysis, we classified how 
the references were translated within a matrix (see Tables 7.4 and 7.5 in 
the Appendix). 

For example, when statements within the texts referred to governmen- 
tal papers authorized by state authorities in the past, we classified them in 
the first column (regulative; structures). In this case, we considered the 
reference use to reflect a formal regulation for how to provide education 
as a service to the population. When the documents made use of argu- 
ments to project expectations about performance and/or to discuss stan- 
dards for how to assess the competence of the future learner, we classified 
them in the fourth column (cognitive; outcomes). In the second and 
third columns we divided sources into arguments that appeared to be 
merely informed by knowledge in the present (the second column: sub- 
stantive; content) and arguments that reflected normative-oriented state- 
ments in the present (third column). Regarding the first column, it is 
important to add that normative statements are considered regulative, 
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while normative arguments in the third column are regarded as axiologi- 
cal (i.e., value-based) as they refer to the soft governing side of ongoing 
activities by expecting learning and change. These alternatives resulted in 
a four-field classification (Sivesind et al., 2016). 

In addition to this categorization, we added the titles and subtitles of 
the chapters in the matrix (Tables 7.4 and 7.5), which helped to create an 
overview of the thematic realms covered by the documents. The table 
includes the titles of only those chapters in the two white papers that 
contained citations of prominent knowledge sources, and the titles are 
our own translations. 

Finally, we conducted a comparative analysis of the translation of refer- 
ences in the two white papers. In the following section, we present the 
argumentative use of co-cited sources and most cited in-text references 
that belong to different categories or levels of evidence: meta-analysis, 
configurational reviews, empirical research studies, as well as official and 
governmental papers. 


A Comparative Analysis of Reference Use 
Meta-analyses 


The following publications were allocated to this category: 


* Hattie (2009) 
° Durlak et al. (2011) 


In correspondence with their characteristics as meta-analyses, the two 
publications refer to studies based on student performance as a bench- 
mark for successful learning and teaching. They present strategies, tech- 
niques, and approaches intended to increase student performance and, in 
their own terms, quality in education. Accordingly, they are mostly trans- 
lated in a purposive way by giving normative directions for teaching and 
learning and projecting future goals. 

While the quotations in WP#1 are more general, those in WP#2 are 
more specific and contain precise results and knowledge from the 
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meta-analysis. With one exception, the quotations present suggestions or 
recommendations that are clearly future-oriented as they refer to increas- 
ing student performance as a criterion for success. The following example 
uses Hattie to legitimize a core feature of the reform (i.e., three specific, 
core curricular goals); as with all quoted examples, the translations are 
our own: 


The department has defined the following three sector goals for basic edu- 

cation, which sum up core elements in both the objects clause within the 

education act and the national curriculum framework: 

* The students shall have a good and inclusive learning environment 

* The students shall master basic skills and have good subject-specific 
competence 

* More students and apprentices shall complete secondary education 
(Years 13—16) 


‘The three goals are connected to each other and sum up the school’s task 
for society. A good and inclusive learning environment is both a goal in 
itself and a tool for increasing the students’ learning outcomes. The objects 
clause points out, among other things, that schools and apprenticeship 
companies should meet students and apprentices with confidence and 
respect while working against all forms of discrimination. All children and 
youth should feel comfortable and be included. Schools that focus on a 
good and indusive learning environment also reach better learning out- 


comes. (Hattie, 2009, Bakken & Seippel, 2012) (W/P#2, pp. 15) 


The paragraph states that a good and inclusive learning environment 
leads to better learning outcomes, which indicates that the matter of stu- 
dent well-being is subordinate to learning outcomes. Since the paragraph 
explicitly refers to learning outcomes, we have categorized this citation as 
reflecting a cognitive and learning-outcome-oriented program. 

The scientific article by Durlak et al. (2011) is cited once in each of the 
two white papers, in both cases for the same argument: that soft skills 
contribute to student performance. The first citation presents knowledge 
on student learning, while the second one argues that soft skills also need 
to be supported from early on. By referring to the increase of student 
performance while subordinating soft skills to it, the reference to this 


7 Structuring School Reform Policy with Evidence... 205 


publication clearly promotes a purposive program that suggests specific 
measures that, in a normative way, puts student performance on top of 
the agenda, as shown by the following example: 


Social and emotional skills like patience, mastering ones own feelings, 
curiosity, and mastering resistance, play an important role in student learn- 
ing. (Backer-Grondahl & Nerde, 2015; Birch & Ladd, 1997; Heckman & 
Kautz, 2013; Durlak et al., 2011) (WP#2, p. 42) 


In both cases, the journal article is referenced together with additional 
sources. The purpose of citing this scientific article is probably to counter 
an often-raised critique against testing frameworks. 

In general, the two white papers draw on meta-analyses in the follow- 
ing subject areas: student learning and the pedagogical work at school 
(WP#1) and quality in education, collaboration, early intervention, and 
competence development in the municipalities (WP#2). By underlining 
knowledge in these thematic areas, the meta-analyses support core points 
of the two white papers, namely, the need to strengthen student learning, 
quality development, collaboration, and competence development on 
the municipality level, even though some of these areas are not covered 
by the main arguments in the publications by Hattie (2009) and Durlak 
et al. (2011). Also striking is that Hattie’s book is not used to argue for 
specific methods in teaching, such as direct instruction or assessment 
practices, even though a “fundamental change in teaching” is announced 
in WP#2. It is more or less used as a justification for strengthening a 
purposive orientation to school reform in general. 


Configurational Reviews 
The following reference was allocated to this category: 
* Nordenbo et al. (2008) 


Nordenbo et al. (2008) are cited surprisingly few times compared to 
other sources in this study. The report is referenced seven times in the 
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whole dataset, whereas it is cited only twice in WP£1 and once in WP#2. 
It serves as a core source for demanding more competence development 
to increase collaboration among teachers, to improve student perfor- 
mance, and to demand more teachers with subject-specific training, espe- 
cially in subjects that do not (yet) require subject-specific education, such 
as art and handicraft education. These demands represent purposive pro- 
grams by anticipating improved student performance, which will follow 
from a set of normative expectations expressed in this way: 


It is thoroughly documented in research that the teachers’ subject matter 
knowledge has an impact on the students’ learning outcomes. Teachers 
who feel confident about their subject matter are less dependent on pre- 
defined teaching designs and methods, and they can make variations and 
develop their own teaching further (Nordenbo et al., 2008). For example, 
it is documented that it is important that teachers possess the practical 
skills chat their students are expected to develop (Espeland, 2011). For this 
reason, the Ministry of Education will evaluate whether one should 
introduce new competence requirements for teaching in several subjects at 
the primary school level. (WP#1, p. 74) 


An interesting aspect in the example above is how the paragraph intro- 
duces Nordenbo et al. (2008). The argument starts with “it is thoroughly 
documented in research,” indicating that Nordenbo et al.’s report is 
indeed considered a significant knowledge source that provides the best 
research evidence, even though the report draws on several references that 
are, in fact, not what is typically regarded as scientific. We categorize all 
three citations as reflecting a purposive policy program, two of them as a 
normative type of reference and one as merely informative. This means 
that Nordenbo et al. (2008) is used to project best practices by creating 
cognitive expectations in one case. For the other two cases, the reference 
is used for a normative purpose, indicating what should be done to 
develop competence among teachers in schools. 

Interestingly, the white papers make in-text references to Nordenbo 
et al. (2008) under the following headings: professions in the school, 
competence development and capacity building, and competence 
requirements for hiring and teaching. In only one case does the reference 
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appear together with another source, which is an expert report about the 
role of the teacher (Dahl et al., 2016). This official report also synthesizes 
education research and is used for partly purposive reasons (five times 
within WP#2 with a purposive aim and two times within the substantive 
category, reflecting a conditional orientation). Although this expert 
report describes the teacher role and elaborates on the need for profes- 
sionalization of teachers, it covers several of the themes in WP#2, such as 
collaboration among teachers, the role of the teacher, and the depart- 
ments assessment of the quality of the evaluation system. Thus, this 
report, which includes some configurational reviews, covers a broader set 
of themes than Nordenbo et al.’s (2008) analysis. This finding shows that 
configurational reviews that focus on certain evidence might serve differ- 
ent functions and purposes than expert reports that are more comprehen- 
sive in terms of the knowledge sources they draw on. Yet, by comparing 
the use of various types of reviews, we find that configurational analyses 
are used less frequently to specify cognitive outcomes than the meta- 
analyses of Hattie (2009) and Durlak et al. (2011). 


Empirical Research Studies 
The following references were allocated to this category: 


° Aasen et al. (2012) 
° Nordahl and Hausstátter (2009) 
* Dale et al. (2011) 


It is perhaps self-evident that the evaluation of the Knowledge 
Promotion Reform plays a crucial role in the two white papers. These 
research-based evaluations were funded by the state and can be classified 
as policy research (Christensen & Holst, 2017). Aasen et al.’s (2012) 
report on the evaluation of the Knowledge Promotion Reform accord- 
ingly has the highest number of citations in the two white papers but is 
co-cited only five times in the whole dataset. Apart from a number of 
citations that present background knowledge in the reports, Aasen et al. 
(2012) is mainly used to legitimize the renewal of the Knowledge 
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Promotion Reform (2016/2020) by showing weaknesses and creating 
options for discussing new issues in the context of the current reform. In 
so doing, it delivers core arguments for the basic measures of the renewal 
of the Knowledge Promotion Reform (2016/2020), including that basic 
skills should be integrated in the subject matter curricula, that goals and 
content need to be more explicit, and that the connection between sub- 
ject matter and the new general part of the curriculum should be strength- 
ened. In addition, Aasen et al. (2012) is used in WP#1 to argue for 
strengthening the on-site work on curricula, activating the political level 
(school owners), clarifying the responsibilities and the relationship 
between different levels in the education system, improving schools’ abil- 
ity to understand and make use of quality assessment results, and pro- 
moting a more decentralized competence development system in which 
municipalities should gain more scope of action. 

This long list of demands shows the impact of the publication and 
reveals that the authors of WP#1 obviously counted on the evidence pro- 
vided by Aasen et al.’s (2012) report. However, most of the quotations 
(12 out of 20) appear in WP#2. These quotations mainly deal with the 
question of how different levels in the municipalities and the schools 
share responsibilities through collaboration and process the information 
of student assessment tests across contexts. This information is used as 
background information and as additional support for the demand for a 
more enhanced system of competence development at schools, as the fol- 
lowing example shows: 


The evaluation of the Knowledge Promotion Reform showed that the 
reform contributed to increasing the quality of basic education, but that 
many small municipalities and schools experienced challenges in imple- 
menting the reform. From the perspective of the county governors, many 
municipalities have gained a stronger grip on the role of school principals 
and the responsibility for schools in their municipality. But the regional 
governor also perceives differences among the municipalities. These differ- 
ences are first of all connected to the size of the municipalities, but also to 
their way of organizing. Small municipalities are in many cases vulnerable 
concerning their economy and competence. Some of them compensate for 
this with an established network for collaboration on the level of the 
municipalities and the schools. (Aasen et al., 2012) (WP#2, p. 31) 
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The citation is not normative as such but describes in a rather open way 
how municipalities have implemented the 2006 reform. It does not sug- 
gest a specific solution but mentions that some municipalities draw on 
networks of collaboration for coping with the challenges they are facing. 
We have categorized the citation as oriented to substantial issues and 
thereby belonging to a conditional type of program. 

In WP#1, however, quotations are explicitly used to justify both spe- 
cific demands and the renewal of the curriculum in general, as below: 


There is research that indicates that the General Part and the Principles for 
Education are part of the local work with curricula only to a small extent 
under the Knowledge Promotion Reform (Aasen et al., 2012). [...] In 
order to create better cohesion in the curriculum framework, the 
Department wants to renew the current General Part, Principles for 
Education, and the subject curricula. This is supposed to contribute to a 
more holistic curriculum framework, updated for today's and the futures 
society. (WP#1, p. 19) 


All in all, we categorized citations from Aasen et al. (2012) either as 
substantially oriented, reflecting a conditional approach, or as normative, 
reflecting a purposive approach. WP#1 uses the research study in a nor- 
mative way, while WP#2 uses it in a more substantive way. This finding 
is an interesting difference because we could have expected the opposite 
pattern. One might interpret this pattern as a dynamic use of evidence: 
in issues that are more conditionally oriented, such as the curriculum, the 
reference points to the purposive side; in thematic areas that are more 
purposive, such as quality development, Aasen et al. (2012) is used to 
strengthen the conditional side. Another explanation could be that the 
temporal order of the two white papers and the mandate framed how the 
policymakers utilized evidence in the two white papers. 

The argument that draws on the reference to Nordahl and Hausstátter 
(2009) is quite inconspicuous. By citing the number of students who are 
in need of support for special needs in the course of their education, 
WP#2 offers insight into conditions for organizing special needs educa- 
tion. We categorized the citation as informative, associated with a condi- 
tional type of program, opposite of a normative or cognitive argument 
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that promotes strategic actions. Both the Aasen et al. (2012) and Nordahl 
and Hausstatter (2009) publications provide national and local knowl- 
edge about the former school reform, the Knowledge Promotion Reform 
of 2006. 

Dale et al. (2011) authored another research report, based on the same 
conditions for conducting research. This report is not among the most 
frequent co-cited publications, but it is one of the most cited in-text ref- 
erences in WP#1. It is cited eight times, four of which are in conjunction 
with Aasen et al. (2012). Only one of the eight citations uses the source 
merely to inform about reform conditions in a substantive way. Instead, 
WP#1 uses the Dale et al. (2011) source to strengthen the purposive 
orientation of the reform proposal. None of the reports in this category 
are, however, used to create narratives or specifications about outcomes 
or best practices, which is also an interesting observation. 


Formal Documents: Governmental and Official Reports 
The following publications were allocated to this category: 


° St. Meld. 30 (2003-2004). Kultur for lering [Culture for learning]. 
(White Paper No. 30 to the Storting). Ministry of Education 
and Research. 

* NOU 2003: 16 J forste rekke: forsterket kvalitet i en grunnopplaring for 
alle [In the First Row. Increased Quality Within a Basic Education 
System for Everyone]. Ihe Committee for Quality in Primary and 
Secondary Education in Norway. 

° St. Meld. 16 (2006-2007) ... og ingen sto igjen. Tidlig innsats for livs- 
lang lering [And No One Is Left Behind. Early Intervention for 
Lifelong Learning]. Ministry of Education and Research. 

* St. Meld. 29 (1994-1995) Om prinsipper og retningslinjer for 10-árig 
grunnskole—ny læreplan [Principles and Guidelines for 10-Year 
Compulsory Schooling—New Curriculum]. Ministry of Education, 
Research and Church Affairs. 

° St. Meld. 20 (2012-2013) På rett vei [On the Right Path]. Ministry of 
Education and Research. 
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° St. Meld. 11 (2008-2009) Lareren—Rollen og utdanningen [The 
Teacher's Role and Education]. Ministry of Education and Research. 

° Ot.prp. nr. 46 (1997-1998) Om lov om grunnskolen og den vidaregåande 
oppleringa (oppleringslova) [Law Draft on Act Relating to Primary 
and Secondary Education and Training (The Education Act)]. 

e NOU: 8 Fremtidens skole: fornyelse av fag og kompetanser [The School 
of the Future: Renewal of Subjects and Competences]. Ministry of 
Education and Research. 

° Dahl, T., Askling, B., Heggen, K., Kulbrandstad, L. I., Lauvdal, T., 
Qvortrup, L., Salvanes, K. G., Skagen, K., Skrøvset, S., Thue, F. W., & 
Mausethagen, S. (2016). Om lærerrollen. Et kunnskapsgrunnlag [On 
the Role of the Teacher. A Knowledge Basis]. Fagbokforlaget. 

° St. Meld. 28 (2015-2016). Fag- Fordypning—Forståelse. En fornyelse 
av Kunnskapsløftet [White Paper: Subjects—In-Depth Learning— 
Understanding. A Renewal of the Knowledge Promotion Reform]. 
Ministry of Education and Research. 


Formal documents make up more than half of the co-cited references 
on the list of the most cited references. They play a crucial role in the 
reform because they serve as a link to previous reforms and arguments for 
reforming education in earlier periods. Mostly they serve as normative 
points of reference, from which specific further steps in the renewal of the 
Knowledge Promotion Reform (2016/2020) are deducted. In this way, 
former written and published official knowledge is credited with legiti- 
mized power that does not need further justification. 

The following quotation is the starting point for arguing for more col- 
laboration among teachers and principals to enhance quality develop- 
ment and student learning: 


One of the ambitions of the Knowledge Promotion Reform was that 
schools should develop a culture for learning to a larger extent. The starting 
point for this goal was the St. Meld. 30 (2003-2004) Culture for Learning, 
which stated that schools have to be learning organizations. The goal was 
that the schools' ability and willingness to learn and develop them further 
should be improved. (WP#2, p. 27) 
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The governmental and official reports establish continuity in the reform 
process by providing a history that connects all previous reforms with 
those that will be implemented. The use of formal documents in the two 
white papers appears to be evenly distributed, with a slight tendency for 
WP#2 to lean more heavily on formal documents. With only a few excep- 
tions, we categorized the use of formal documents as regulative, reflecting 
a conditional program for reforming education and schooling; as such, 
they stabilize the reform rather than renewing and transforming the 
schools in radical terms. 


Conclusions 


In this chapter, we analyzed the distribution and argumentative use of 
co-cited and in-text references within and across policy realms of Norway's 
most recent school reform. We aimed to examine how policymakers and 
experts responsible for writing two white papers and eight public inquiry 
reports responded to critiques and trends that have generated demands 
for a more evidence-based policy. We asked how state administrative bod- 
ies such as ministries and inquiry bodies made policy evidence-based by 
referencing different types of knowledge sources and how their modes of 
arguing for reform translated knowledge in terms of two decision pro- 
grams that each covers two types of arguments. As a result, we assessed 
how references were embedded in regulative, informative, normative, or 
cognitive arguments. 

Both our bibliometric network and semantical analyses revealed that 
the authors and bodies involved in this work have reacted to the critique 
that inquiries have been too practical and general in earlier reform peri- 
ods. This reaction is documented in the extensive usage of references 
within the white and green papers. Throughout this chapter, we have 
demonstrated that Norwegian reform documents authorized by the 
Ministry of Education are thereby heavily evidence-based, as shown 
through their references to various types of knowledge (e.g., research 
reviews including meta-analyses and configurational analyses) and their 
use of phrases like “as thoroughly documented in research.” The core 
basis of the most cited documents within the full database contains not 
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only typical evidence-based reviews, but also many governmental docu- 
ments; in fact, more than half of the most co-cited references are govern- 
mental reports or documents. In addition, by taking in-text references 
into consideration, we showed that the report of Aasen et al. (2012) plays 
a crucial role by setting the scene for reform and renewal across a variety 
of topics within both policy realms related to the curriculum reform and 
to educational governance of the basic education system. 

The argumentative use of the references in the two white papers cor- 
responds, more or less, to the type of category they represent. In other 
words, reviews in the form of meta-analyses and configurational analyses 
tend to be used in a purposive manner, while governmental papers are 
mainly used to ensure that certain conditions are in place and formal 
procedures are followed (conditional manner). However, some observa- 
tions show interesting variations and unexpected translations. 

First, there is a commonly held view that reform options become stan- 
dardized through evidence-based policy that makes use of measurements 
as quantifiable expressions of performance. In our study, relatively few 
citations specify standards for best practices (see "cognitive outcomes" on 
the right side of Tables 7.4 and 7.5). Few publications that were cited in 
the white papers provide measures and standards for how to improve 
student learning and outcomes by purpose. The absence of knowledge 
provided by international organizations such as the Organisation for 
Economic Co-operation and Development (OECD) among the most 
co-cited references is possibly a reason for this pattern. However, another 
important finding is that the scientific reviews are in no case used to regu- 
late or inform about past policies or practices in a conditional form, but 
only in terms of arguments that call for purposive, future-oriented 
changes within the education system. Thus, the use of scientific reviews 
directs attention toward performance-oriented outcomes. 

Our analysis shows that meta-analyses and configurational analyses 
serve as sources for normative recommendations for how to improve stu- 
dent learning instead of explicitly referring to results of performance 
studies or specifying cognitive outcomes. This finding holds particularly 
true for WP#1, whereas WP#2 references Hattie (2009) five times to 
specify cognitive outcomes. Interestingly, another important finding is 
that the scientific reviews are in no case used in a conditional way; instead, 
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they are used as arguments for purposive, future-oriented changes within 
the education system. 

Second, our analysis shows that there is significant use of governmen- 
tal reports for regulative use, which means that arguments in the two 
white papers refer to formal procedures or decisions documented in those 
governmental reports. The use of formal documents seems to help with 
identifying the intention of past decision processes. In addition, they sta- 
bilize the reform by creating memories of the past. However, in WP#1, 
governmental reports are used not only in a regulative manner, but also 
in substantive, normative, and cognitive arguments. The use of NOU, 
2015: 8 is the prime explanation for this pattern, as it is used frequently 
to legitimize both conditional and purposive program forms. This source 
is the only one among the most prominent references that is used to 
make all four types of arguments (i.e., regulative, informative, normative, 
and cognitive). A dominant pattern is that NOU, 2015: 8 serves a pur- 
posive role to legitimize the renewal of the national curriculum. The 
report was cited in chapters like “competence aims within the subjects,” 
"an increased cross-disciplinary orientation in teaching and learning," 
"strengthening clearer priorities within the subjects" "emphasizing 
improved coherence and progression,” “learning strategies and reflection 
on ones own learning," and "competence development and capacity 
building" (see Table 7.4). The pattern is in no way surprising, as the 
authors of WP£1 were mandated to legitimate the renewal of the reform 
by drawing on earlier governmental and official reports, particularly 
NOU, 2015: 8 The School of the Future. The document, however, is not 
included as a reference in WP#2, which addresses issues related to early 
intervention and quality in schools. 

Third, we found that the program evaluation reports that contribute 
empirical research on education reforms are used for various purposes. 
Interestingly, Aasen et al. (2012) is distributed over two categories, reflect- 
ing both substantive and normative forms of use. While it tends to be 
translated in a normative-oriented way in WP#1 in tandem with the 
Dale et al. (2011) report, authors of the WP#2 apply Aasen et al. (2012) 
in a substantive, retrospective way. In WP#2, the use of Aasen et al. 
(2012) differs from references to Dahl et al. (2016), which was formally 
commissioned by the state as an expert report. Dahl et al. (2016) 
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summarized knowledge about the teacher as a key agent within the edu- 
cation system, for which it is used in a normative way for several themes, 
such as "teachers" collaborations in school.” For the sections that cover 
quality assessment and the evaluation system, neither the reference to 
Aasen et al. (2012) nor Dahl et al. (2016) sets the direction in a purposive 
way (see Table 7.5). 

Finally, the reference patterns we have uncovered through our analysis 
help us explore the role of evidence in education policy. Evidence seems 
to be structured by the knowledge sources that are used while being 
deeply dependent on the arguments formulated by policymakers. 
Looking for examples in the content of the references, we see a clear pat- 
tern in which formal documents or governmental papers connect reform 
options with political ambitions and jurisdictions. These connections 
exist because the formal documents offer information about various con- 
ditions for organizing education and because they are mandated to do so. 
Moreover, empirical research studies and reviews that refer to the inner 
life of schools can be used to inform policymakers about educational 
concerns in renewing teaching and learning. Thus, these references are 
affiliated with topics that policymakers potentially address when they 
explore options for education reform. If they are also written under the 
auspices of state-funded programs to evaluate the reform, this condition 
is likewise an explanation for their prominence. We discovered that the 
two white papers frequently cite evaluation reports about the former cur- 
riculum reform, the Knowledge Promotion Reform 2006. This finding 
underlines how the evaluation and use of research reports enable a tight 
connection between the previous and the current reform. Apparently, 
results and interpretations from the evaluation. of the Knowledge 
Promotion Reform inform policymakers; more importantly, they serve as 
a normative basis for formulating specific policy options that aim at cur- 
riculum renewal and improvement. 

No matter their intended use, references serve the function of bringing 
evidence into policy, in terms of either bibliometric networks or explicit 
use to formulate arguments within the bodies of the texts. Through this 
function, references align systems of reasoning that are highly differenti- 
ated in practice. In this way, policy knowledge plays an inter-mediational 
role by shaping reform options connecting both policy realms and sys- 
tems of reasoning. 
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WP 20 (2012) QaD = St. Meld. 20 (2012—2013). På rett vei. Kvalitet 
og mangfold i fellesskolen. [On the Right Path. Quality and Diversity in 
the “School for All”] (White Paper No. 20 to the Storting). Ministry of 
Education and Research. 

WP 29 (1994) PaG = St. Meld. 29 (1994-1995). Om prinsipper og 
retningslinjer for 10-årig grunnskole—ny læreplan [Principles and 
Guidelines for 10-Year Compulsory Schooling—New Curriculum] 
(White Paper No. 29 to the Storting). Ministry of Education, Research, 
and Church Affairs. 

WP 30 (2003) CfL = St. Meld. 30 (2003-2004). Kultur for lering 
[Culture for learning] (White Paper No. 30 to the Storting). Ministry of 
Education and Research. 

WP 20 (2012) QaD = St. Meld. 20 (2012-2013). Pa rett vei. Kvalitet 
og mangfold i fellesskolen. [On the Right Path. Quality and Diversity in 
the “School for All”] (White Paper No. 20 to the Storting). Ministry of 
Education and Research. 

WP 28 (2015) CR = St. Meld. 28 (2015-2016). Fag— Fordypning— 
Forståelse. En fornyelse av Kunnskapsløftet [Subjects—In-Depth Learning— 
Understanding. A Renewal of the Knowledge Promotion Reform] (White 
Paper No. 28 to the Storting). Ministry of Education and Research. 

WP 29 (1994) PaG = St. Meld. 29 (1994-1995). Om prinsipper og 
retningslinjer for 10-årig grunnskole—ny læreplan [Principles and 
Guidelines for 10-Year Compulsory Schooling—New Curriculum] 
(White Paper No. 29 to the Storting). Ministry of Education, Research, 
and Church Affairs. 

WP 30 (2003) CfL = St. Meld. 30 (2003-2004). Kultur for lering 
[Culture for learning] (White Paper No. 30 to the Storting). Ministry of 
Education and Research. 


How to Read the Matrices in Table 7.4 and 7.5 


Columns: 

The column on the left lists the titles of chapters and sub-chapters of 
the white papers in which we found references from our database in the 
text. The next four columns refer to our analytical categories based on 
Luhmann. Conditional and purposive programs are divided into 
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regulative and substantive forms on the one side and normative and cog- 
nitive forms on the other side. 

Rows: 

Each of our references stands for a specific type of knowledge, which 
we highlighted in different shades of gray (see also Table 7.2 in the text). 


Meta-analysis Ex.: Hattie (2009) 

Configurational Ex.: Nordenbo et al. (2008) 
analysis 

Empirical research Ex.: Aasen et al. (2012) 
study 


Governmental paper Ex.: WP 28 (2015) CR (=White Paper St.M. 28, 
curriculum renewal) 


References written in italics are in-text references, while references 
written in ordinary style originate from the list of the most cited 
documents. 

Each reference was assigned a program form based on the framework 
of Luhmann (2000) and Sivesind et al. (2016) according to its use in the 
text of the white paper. 


Notes 


1. A similar analysis has already been published by Baek et al. (2018) and 
Steiner-Khamsi et al. (2020) drawing on a slightly different database. For 
the analysis in this publication, only official reports have been included 
that explicitly referred to education and schooling. 

2. For a full overview of source documents, see Chap. 2. 
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The Complexity of Context 
in Legitimating National School 
Reforms: The Case of Sweden 


Andreas Nordin and Ninni Wahlstróm 


A Theoretical Background 


Policy research has convincingly demonstrated the significance of the role 
of policy entrepreneurs, who bring in new ideas for how to define and 
solve policy problems, in policymaking processes (e.g., Verger, 2012). 
One of the resources that policy entrepreneurs draw upon is empirical 
research. However, entrepreneurs often advocate for certain ideologies 
and perspectives, using research in a strategic way to promote a particular 
solution. The identity and ideology of the knowledge entrepreneur also 
matter, including whether they are involved in a particular political party, 
a certain think tank, or a non-governmental organization or whether they 
work as a researcher at a university. According to McDonnell and 
Weatherford (2013), research-based evidence is likely to be used in com- 
bination with other evidence in line with the elected officials’ core values 
that evoke their positive response. For example, Kingdon (1995) noted 
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that policymaking and agenda setting are not linear processes, starting 
with the problem formulation and ending with a policy decision for a 
solution. Instead, policymaking is characterized by parallel streams of 
politics and policies, with the implication that a solution can very well be 
preferred and suggested before a more specified policy problem is 
formulated. 

This chapter draws on two bodies of related research from comparative 
and international education informing the research questions and mak- 
ing up the interpretative framework to examine the complexity of both 
layers and sequencing. The two bodies of research are the role of networks 
in the shift from government to governance and the study of "traveling 
reform" in diffusion versus reception studies. 

The general shift from government to governance in Western countries 
is commonly seen as the result of new public management policies that 
most OECD countries introduced in the wake of the neoliberal reforms 
of the 1980s and 1990s. In this shift, policy networks play a prominent 
role (Lubienski, 2019). In the education sector, this shift created a new 
role for the state, new ways of regulating the education system, and new 
tools for generating, or alleviating, reform pressure. In turn, the outcomes 
of new public management reform projects triggered a proliferation of 
standardized student assessments. International comparisons (including 
international large-scale student assessments such as PISA, Trends in 
International Mathematics and Science Study [TIMSS], and Ihe OECD 
Teaching and Learning Study [TALIS]) have since been referenced as key 
sources in policymaking for a variety of reasons. Governments have used 
them as primary monitoring tools to assess the quality of their teachers, 
schools, districts, and education systems, as well as to make policy deci- 
sions based on these assessments (Addey et al., 2017; Lingard & Sellar, 
2016). The shift from government to governance has fueled a “gover- 
nance by numbers" (Grek, 2008; Lindblad et al., 2018; Molstad & 
Pettersson, 2019) approach, while also empowering non-state actors to 
participate in the new millennium as key policy actors. This has been 
interpreted as a clear sign of the “disarticulation and diversification of the 
state system," the “destatalisation” of the policy process (Ball & Junemann, 
2012, p. 24), and a changing role for the state where the state has handed 
over previous national influences and responsibilities to partly new arenas 
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such as international organizations and private actors (Ozga & Lingard, 
2007; Wahlstróm, 2014). Strikingly, the interplay between the state and 
local actors does not necessarily follow formal decision routes where the 
state delegates professional responsibility to its civil servants. Instead, at 
times, the state actively intervenes in the business of local practices 
through the use of evidence, formal standard setting, and sanctions 
(Wahlstróm & Sundberg, 2018). 

We also draw on research on traveling reforms, which includes two 
different perspectives. Researchers who adopt a bird's-eye view on the dis- 
semination of international standards or global scripts in education take 
their point of departure from the global or the world perspective. This 
perspective enables an understanding of why and how some policies 
become scaled up and spread while others do not. Such a perspective 
allows for an understanding of the active role of international organiza- 
tions in lending or disseminating specific policies and programs, coined 
as “best practices" or “international standards” (Bromley & Meyer, 2015; 
Krücken & Drori, 2010). Such a “perspective from above,” however, pro- 
vides only one of several possible angles for understanding the spread of 
global education policy. By contrast, a "perspective from below" illumi- 
nates why, how, and when national or local policy actors selectively bor- 
row global education policies (Anderson-Levitt, 2003; Sivesind & 
Wahlström, 2016; Steiner-Khamsi, 2012; Steiner-Khamsi et al., 2020; 
Wahlström, 2020). In these studies, the researchers examine which ele- 
ments of international policy discourses will be of interest to national 
policy actors to interpret and reconstruct for the legitimization of 
national/local policy needs. The researchers examine the way global poli- 
cies are used for legitimate and/or delegitimate purposes when recontex- 
tualized into national policy agendas. 

In this study, the perspective is "from below." From a national level, we 
explore how the Swedish government makes use of various knowledge 
sources of evidence for legitimating its reform recommendations. We are 
interested in the various international, regional, and national policy and 
knowledge actors that the government turns to for providing evidence for 
its recommendations for decisions. 
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Purpose and Research Questions 


From this point of departure, the aim of this study is to advance the 
understanding of the mechanisms at stake in legitimating national school 
reforms. More precisely, we examine the evidence used by the Swedish 
government in the 2015/2018 Knowledge Achievement Reform as mani- 
fested in the reference lists of the green papers (GPs) and white papers 
(WPs) analyzed. For this purpose, the following research questions are 
relevant: 


* Whose knowledge is used to legitimate the 2015/2018 Knowledge 
Achievement Reform? 

° What types of knowledge sources are used to legitimate the 2015/2018 
Knowledge Achievement Reform? 

* How much expert knowledge is translated upward into political 
knowledge? 


Contextualizing the Swedish Decision-Making 
Process at the State Level 


Over a long period of time, Sweden has appointed commissions as a way 
for governments to produce evidence for political decision-making and 
for the "anchoring" of policy recommendations in different segments of 
Swedish society. The Swedish system stands out because of its strong 
emphasis on the preparatory stages. Commissions of inquiry have two 
distinguishing features. First, they are appointed by the Cabinet and have 
a legal status similar to other government bodies. Second, commissions 
of inquiry are set up with a mandate to investigate a special thematic area 
or particular subject (Petersson, 2016). This institutional arrangement is 
not inscribed in Swedish law but has evolved as a praxis over a long period 
of time. Almost all pieces of legislation are prepared in this way, which 
means that around 200—300 governmental commissions are at work at 
any given time in the political system. By international standards, Swedish 
ministries are quite small; therefore, appointing commissions may also be 
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a way to temporarily expand the staff of the ministry outside the ordinary 
budget. Once constituted, the commission has a considerable degree of 
autonomy. Although efforts have been made to limit both the size of the 
commissions and the time period for their work, it is still not unusual for 
their work to extend over several years (Trägårdh, 2007). Their work is 
finally presented and handed over to the responsible minister in the for- 
mat of government official reports (green papers, known by the Swedish 
acronym SOU). Appointing commissions of inquiry has to do with the 
political relationship between the state and civil society. These commis- 
sions have served as a “linchpin in a system of democratic governance 
whose hallmark is deliberative political practices that involve a mix of 
civil servants, politicians, academics, experts, and representatives of rele- 
vant civil society organizations” (Tragardh, 2007). As Tragardh (2007) 
noted, this extended communicative law-making process also contributes 
to societal legitimacy and acceptance for a new law or policy and adds 
trust to the system. 

Today, the timespan of the policymaking process has become shorter 
compared to the praxis during the mid-twentieth century. In the 2010s, 
the commissions of inquiry were more tightly controlled by the Cabinet 
and government ministries, while parliamentary representation declined. 
While the level of striving for consensus in the inquiry process has 
decreased, the level of conflicts has increased and the politics in parlia- 
ment have become more polarized (Petersson, 2016). Moreover, in recent 
years, Swedes have demonstrated a decline in their trust of the political 
system as a whole, with a widened gap between the voters and the elected 
as a result (Amná, 2006). In addition, Gunnarsson et al. (1998) have 
shown a shift in the way commissions were composed between 1988 and 
1997, with a decrease in the number of larger commissions with a chair- 
man in favor of one-person commissions with additional members and 
experts linked to the investigation. This development has continued. In a 
study analyzing the period 1990-2016, Dahlström et al. (2019) showed 
that 9096 of the ongoing commissions of inquiry in 2016 were in the 
form of one-person commissions. At the same time, barely 396 consisted 
of parliamentary committees of inquiry. Dahlstróm et al. (2019) con- 
cluded that the kind of committees that once constituted the backbone 
of the committee system (i.e., broad parliamentary committees) have 
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almost disappeared today. We understand this development as an expres- 
sion of trying to rationalize and speed up the process of national policy- 
making in the context of increased global competition (see Kamens, 
2013; Lewis & Hogan, 2019). 


A History of Borrowing 


Swedish school reforms have a well-established history of borrowing pol- 
icy from other educational systems, particularly for policy domains 
related to curriculum and governing of schools (Hallsén & Nordin, 
2018). The Governmental School Commission of 1946 talked about the 
Swedish system of whole-class teaching as being influenced by “German 
school life" (SOU 1948:27, p. 112). The same report also revealed that 
members of the commission had taken study visits to Denmark, Norway, 
Belgium, France, Switzerland, the United Kingdom, and the United 
States. During the period leading up to the realization of a nine-year 
comprehensive school for all children in 1962, Sweden also became an 
influential policy lender, with Stockholm as the leading marketizer of the 
new Swedish school system that attracted visitors from all over the world 
(Hallsén & Nordin, 2018). Educational reforms were thoroughly planned 
in terms of time and resources, and school experiments and investigations 
were often led by researchers and described as an expression of social 
engineering (Román et al., 2015). Similarly, in the SOU that preceded 
the curriculum reform of 1980, The Schools Inner Working (SOU 
1974:53), an entire chapter is devoted to international examples of ways 
to improve a school’s work environment, with lessons drawn from 
England, Eastern Europe, and Denmark. The report School for Bildung 
(SOU 1992:94) featured a shift away from the country-specific refer- 
ences with talk in terms of a wider ongoing “internationalization,” of 
which Sweden is part, and references to specific countries as part of orga- 
nizations such as the United Nations (UN) or the European Community. 
From 1995 on, following Sweden’s membership in the European Union 
(EU), education policy in the country has been elevated to a priority on 
the reform agenda. Even though Swedish policymakers have refrained 
from making explicit references to EU policy, developments in other EU 
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countries have had a visible impact on school reform in Sweden, which 
has led scholars to talk about the Swedish strategy as a "silent policy bor- 
rowing" from the wider European education policy space (see Nordin, 
2012; Waldow, 2009). In recent years, the OECD has emerged as an 
increasingly important source of expertise and authorization for the 
Swedish government when making reforms. Ihe OECD today functions 
as an authoritative reference system in justifying borrowing of interna- 
tional standards created in PISA or national OECD expert reports (Grek, 
2019; Ringarp & Waldow, 2016; Wahlstróm, 2018; Waldow, 2012). 


The 2015/2018 Knowledge Achievement Reform 


In 2006, the Swedish government decided to appoint a one-person com- 
mission led by a civil servant to investigate the implementation of the 
new goal-oriented curriculum launched in 1994. In the report that was 
published in 2007 (SOU 2007:28), the investigator concluded that the 
new public management of the Swedish school system along managerial 
principles and decentralized power had not worked out as hoped for. This 
finding led to the launch of a whole range of fundamental reforms in 
2011, including a new school law, a new national teacher education, a 
new curriculum for the comprehensive school, and a new, more differen- 
tiated grading system, all to enhance monitoring and central control in a 
decentralized school system. In addition, Sweden at the time suffered 
from growing concern over continuously declining PISA results. When 
Sweden received its lowest PISA results ever in 2012, the national school 
crisis was a fact (Wahlstróm, 2018) and fueled the national crisis dis- 
course and an emerging “scandalization” (Steiner-Khamsi, 2003) of the 
comprehensive school within politics and the media (Nordin, 2012, 
2019). Against this backdrop, the Swedish government decided to take 
the unconventional measure to turn to the OECD for help (Pettersson 
et al., 2017). After a period of analyzing and visiting Sweden, the OECD 
published a final report in 2015. The OECD stated that more efforts 
were required to fulfill Swedens commitment to excellence and equity 
(OECD, 2015) if the country wanted to come to terms with its national 
problems. In the report, the OECD suggested a focus on the following 
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three areas: (a) promoting quality with equity across Swedish schools, (b) 
building a high-quality teaching profession, and (c) steering policy and 
accountability focused on improvement. To address the OECD report 
and further refine the reforms of 2011, in April 2015, the Swedish gov- 
ernment appointed a government commission called the 2015 School 
Commission, chaired by the Director-General for the Swedish National 
Agency for Education. During the last six months of the commissions 
work, a researcher took over the chair position because the former chair 
was appointed Minister of Education. The 2015 School Commission was 
thus close to the government. 

The commission presented its suggestions in SOU 2017:35 entitled A 
Gathering for School—A National Strategy for Knowledge and Equivalence 
(Source Document 2). In the white paper 2017/18:182 (Source 
Document 1) that paved the way for the 2015/2018 Knowledge 
Achievement Reform, the SOU plays an important role and is explicitly 
mentioned together with SOU 2016:94 (Source Document 7) and SOU 
2017:51 (Source Document 8) as central to the law-making recommen- 
dations formulated in the white paper. The OECD thus played an impor- 
tant role as policy adviser (Lingard & Sellar, 2016) in setting the agenda 
for the Swedish reform, operating as a “boundary organisation” (Grek, 
2019) that pointed out what policy areas to prioritize and indirectly what 
expert knowledge would be of most use. 


Methodological Considerations 


We have pursued the research questions by analyzing references made in 
published policy documents. The bibliometric network analysis is focused 
on “official knowledge” (see Baek et al., 2017) or texts produced by the 
Ministry of Education and Research (white papers) and by its appointed 
commissions (green papers). Since the white paper had no reference list, 
we began by searching through the entire volume, resulting in a total of 
12 cited green papers of which one lacked a reference list. Of the 12 
documents, two documents (SOU 2016:38, an interim document, and 
SOU 2016:38, the same document as the selected green paper [GP] but 
published two days earlier) were duplicates and therefore removed. 
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Another two documents (SOU 2016:77 and SOU 2017:49) were not 
direcdy related to compulsory education. Thus, the delimitation was 
made due to (a) duplicate documents and (b) documents not related to 
compulsory education. We then ended up with eight green papers that 
served as source documents of which one lacked a reference list. In total, 
we have nine (eight green papers and one white paper) source documents 
for which we have first entered and analyzed a total of 1615 references. In 
the second step, we adjusted the number of references so that references 
cited by multiple sources were counted only once, resulting in a total of 
1421 references. Table 8.1 illustrates the source documents and the dis- 
tribution of references. 

In addition to the quantitative analyses of citation frequency, we have 
classified the reference distribution according to location and type of 
document. The location category distinguishes between domestic, 
regional, and international references, where regional refers to the Nordic 
context. The type of document distinguishes between the following sub- 
categories: reports, books, academic publications, governmental publica- 
tions, and others. 

Each reference has been given a unique identification number, creating 
a matrix for calculating in-degree centrality and co-citations in order to 
understand the reference network of the 2015/2018 Knowledge 
Achievement Reform. We used the software programs UCINET 6.627 
and NetDraw for analysis and visualization of the data (Borgatti 
et al., 2002).' 


The Bibliometric Network Analysis: Findings 


The bibliometric network analysis yields several interesting patterns 
related to the three research questions presented in the introduction of 
this chapter. 


236 


A. Nordin and N. Wahlstróm 


Table 8.1 Distribution of references in the policy documents of the 2015/2018 
Knowledge Achievement Reform 


ID Type Title 
WP White paper 2017/18:182 Gathering for School [Prop. 


1 


2 


GP 


GP 


GP 


GP 


GP 


GP 


GP 


GP 


2017/18:182 Samling fór skolan] 

SOU 2017:35 Gathering for School—National Strategy for 
Knowledge and Equality. Final Report from the 2015 
School Commission [Samling fór skolan—Nationell 
strategi fór kunskap och likvárdighet. Slutbetánkande 
av 2015 árs skolkommission] 

SOU 2008:52 Certification and Stricter Eligibility Rules 
[Legitimation och skárpta behórighetsregler] 

SOU 2013:56 The Independent Schools in Society 
[Friskolorna i samhället] 

SOU 2015:22 The Principal and the Steering Chain. 
Report from the Commission of Inquiry into the 
Principal's Work Situation in the School System [Rektorn 
och styrkedjan. Betánkande av utredningen om 
rektorernas arbetssituation inom skolvásendet] 

SOU 2016:59 On Good Grounds—An Action Guarantee 
for Reading, Writing and Math. Report of the Inquiry 
into a Read-Write-Count-Guarantee [Pa goda 
grunder—en átgárdsgaranti fór lásning, skrivning och 
matematik. Betánkande av Utredningen om en 
Lása-skriva-rákna-garanti] 

SOU 2016:94 Missing! Pay Attention to the Students' 
Absence and Take Action. Report of Turning 
Absenteeism to Attendance—An Investigation into 
Problematic Student Absenteeism [Saknad! 
Uppmárksamma elevers fránvaro och agera. 
Betánkande av Att vánda fránvaro till nárvaro—en 
utredning om problematisk elevfránvaro] 

SOU 2017:51 Education, Teaching and Management— 
Reform Care in Support of a Better School [Utbildning, 
undervisning och ledning—reformvard till stód for en 
báttre skola] 

SOU 2016:66 That Is Correct! Increased Transparency and 
More Equal Conditions [Det stámmer! Ókad 
transparens och mer lika villkor] 


References 
86 


337 


234 


334 


156 


N/A 


Note: ID = Identification number of included source documents; WP = white 
paper; GP = green paper 
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Reference Patterns in the Swedish School Reform 


Bearing in mind the central role of the OECD in the launch of the 
2015/2018 Knowledge Achievement Reform, it is interesting to see that 
as much as 8096 of the references are domestic, mostly referring to vari- 
ous types of laws and regulations on how to govern the Swedish school 
system at large. Of the remaining references, 1996 are international, and 
only 196 are references to other countries in the Nordic region. As shown 
in Table 8.2, Sweden thus distinguishes itself both in terms of the amount 
of domestic references and in the lack of references to the other Nordic 
countries. 

Sweden is an active member of the UN, so the UN Convention on the 
Rights of the Child has played an important role in Swedish policymak- 
ing ever since the parliament ratified it without any reservations in 1990. 
This importance is evident in the results where the UN Convention on 
the Rights of the Child is the most cited international document contain- 
ing regulations. Recently, the parliament has decided to make the UN 
Convention on the Rights of the Child Swedish law from 2020, increas- 
ing its significance even more. Looking at the international references in 
total, the OECD stands out as the most frequently cited. Ihe OECD is 
the fifth most cited publisher after the Swedish government, the National 


Table 8.2 Reference distribution 


Location Types of Documents 
Journal 

ID Total Domestic Regional Int'l Report Book Articles Gov't Others 
1 86 86% 0% 14% 7% 0% 096 77% 16% 
2 337 74% 3% 23% 11% 7% 9% 62% 11% 
3 230 85% 2% 13% 17% 14% 0% 52% 17% 
4 169 96% 0% 4% 0% 2% 0% 90% 8% 
5 69 84% 0% 16% 13% 19% 4% 42% 22% 
6 234 67% 1% 32% 8% 12% 22% 47% 12% 
7 334 81% 1% 18% 17% 4% 13% 50% 15% 
8 156 92% 0% 8% 4% 4% 0% 84% 8% 
9 NA 

Total 1421 80% 1% 19% 11% 8% 9% 58% 14% 


Note: References that are cited by multiple sources are counted only once. 
ID = identification number; Int'l = international; Gov't = governmental 
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Agency of Education, the parliament, and the school inspectorate, in 
that order. 

Another distinct feature of the reform is the limited use of academic 
publications; in fact, the white paper and three of the green papers do not 
contain a single academic reference. References to scientific literature are 
made in the format of synthesized knowledge. Hattie’s publication Visible 
Learning stands out as the most cited document of all, together with 
Swedish school law. Another research review that has had an impact is the 
book Excellent Teaching by Hakansson and Sundberg. Table 8.3 shows 
the distribution of the most cited documents in the Swedish reform. 

Looking at the level of authors, it is interesting to see that the most 
cited researcher is Professor Jan-Eric Gustafsson, who became the chair of 
the 2015 Swedish School Commission after the Director-General for the 
Swedish National Agency for Education (Skolverket) Anna Ekstróm, who 
was the original chair, was appointed Minister of Education and Research. 
Professor Jan-Eric Gustafsson has a total of 11 citations, followed by 
Linda Darling-Hammond with seven citations and John Hattie with six 
citations. The commission led by Professor Gustafsson wrote the green 
paper SOU 2017:35 Gathering for School—National Strategy for Knowledge 


Table 8.3 Most cited documents 


ID Count Title Location Type 


1008 6 Prop. 2009/10:165 Den nya skollagen—fór 1 4 
kunskap, valfrihet och trygghet [The New 
School Law—For Knowledge, Choice and 


Security] 
1287 6 Visible Learning: A Synthesis of Over 800 3 2 
Meta-Analyses Relating to Achievement 
1064 5 SFS 1993:100 Hógskolefórordning [Higher 1 4 
Education Ordinance] 
1085 5 SFS 2011:185 Skolfórordning [School Ordinance] 1 4 
1086 5 SFS 2010:800 Skollag [The Education Act] 1 4 
1066 4 SFS 1998:1474 Kommittéfórordning [Committees 1 4 
Ordinance] 
1275 4 The UN Convention on the Rights of the Child 3 5 
1289 4 Utmárkt undervisning [Excellent Teaching] 1 2 
1525 4 Prop. 1990/91:18 Om ansvaret fór skolan [The 1 4 


School's Responsibility] 


Note: Cutoff point: minimum four times. Location: 1 = domestic; 3 = international. 
Type: 2 = book; 4 = report; 5 = other publications 
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and Equality: Final Report from the 2015 School Commission (Source 
Document 2), which was central to the 2015/2018 Knowledge 
Achievement Reform. Even though an entire commission is responsible 
for writing a green paper, this result shows the importance of the chair of 
a commission (together with its secretariat) in setting the direction and, 
in this case, filtering the kinds of references to be included (see Christensen 
& Holst, 2017). In the Swedish reform, where only 9% of the references 
were academic, even a small number of references makes a significant dif- 
ference in the percentage of the total number of references in that 
category. 


The Structure of the Evidence Base 


The network design of the study also enables an examination of the rela- 
tions between the expert commissions included in this study in terms of 
shared knowledge, reflected as explicit references in the source docu- 
ments. As Fig. 8.1 shows, the overall picture is that the various commis- 
sions have produced highly specialized knowledge in their reports. 


7 
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Fig. 8.1 Complete network structure. (Note: Regional = gray; domestic = white; 
international = black; source = circle; node size = in-degree centrality) 
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The references in Fig. 8.1 are colored based on their domain, where 
domestic references are white, regional references are gray, and interna- 
tional references are black. The size of the nodes indicates the documents 
in-degree centrality, that is, a documents centrality in the network based 
on how many texts refer to that specific document. 

The colors illustrate the dominance of domestic references and the lack 
of Nordic ones in the Swedish reform. The white paper (WP) (Source 
Document 1) is marked as number 1 in the middle of the network struc- 
ture. The largest field, located above the central point constituted by the 
WP, is GP 2, Gathering for School (Source Document 2), which represents 
the commission of inquiry most directly responsible for preparing a suc- 
ceeding recommendation for a reform and also bears the same name in 
Swedish as the WP. Among the shared references, most are domestic gov- 
ernmental references, in line with the overall picture. However, two inter- 
national references stand out as cited by many. The most cited one is 
Hatties book Visible Learning, which is cited six times and marked as a 
black square with the number 1287. Its size also signals importance in 
terms of shared knowledge, as many cite this reference. The other inter- 
national reference that stands out is the UN Convention on the Rights of 
the Child, marked as a black square with the number 1275 placed on the 
left side of Hattie in the network. The references to the UN Convention 
on the Rights of the Child are part of a larger discourse in Swedish school 
governing documents, where the UN Convention on the Rights of the 
Child has played an important role for a long time. Since January 2020, 
it has also become part of national law. 

Also characteristic of the Swedish reform is the limited use of academic 
references. Out of eight source documents, four do not have a single ref- 
erence to academic research. As previously mentioned, when referring to 
research, it is predominantly in the format of synthesized knowledge. In 
the books authored by Hattie and Hakansson and Sundberg, research 
knowledge is collected, interpreted, and presented in a supposedly more 
manageable and accessible format, making them “intermediaries” 
(Lubienski, 2019) between science and politics in providing policymak- 
ers with seemingly "useful" evidence. 
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Fig. 8.2 Complete network structure without the white paper 


Removing the white paper from the network structure shows the 
important role of GP 2 Gathering for School (Source Document 2). The 
field representing this GP is located in the middle of Fig. 8.2. 

GP 2 has mutual references primarily with Source Documents 6, 7, 
and 8. Characteristic for these three GPs is the fact that they were all 
produced after the 2015 OECD report with themes in line with the 
OECD’s recommendations for improving quality and knowledge results 
in Swedish schools. However, the distribution of references does not 
reflect this thematic relationship. While GP 6 (Source Document 6) on 
the action guarantee for reading, writing, and mathematics has 3296 
international references, GP 7 (Source Document 7) on students absences 
has 8196 domestic references and 1896 international references. Likewise, 
GP 8 (Source Document 8) on the adaptation of rules for teacher certifi- 
cation has 9296 domestic references, 896 international, and not a single 
reference to other Nordic countries. 'This result adds to the understand- 
ing of commissions operating first and foremost on a “pragmatic man- 
date" generating “highly specialized knowledge on the specific topic for 
which they are tasked" (Steiner-Khamsi et al., 2020, p. 128), which can 
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result in radically different reference structures in SOUs, even when 
related as parts of a wider shared discursive context. 


Upward Translation in the Swedish School Reform 


Turning to the process of translating expert knowledge upward, the result 
shows a selective use of expert knowledge at the political level. 

Figure 8.3 demonstrates that the white paper (Source Document 1) by 
the Swedish Ministry of Education and Research shares about 4496 of the 
knowledge produced by the experts in the commissions of inquiry that it 
has appointed, while about 5696 is unique to the white paper. As such, 
more than half of the number of references is lost in the political process 
of "upward translation" (Steiner-Khamsi et al., 2020). Additionally, this 
finding indicates that the Ministry of Education and Research looked for 
other sources of expert knowledge apart from the green papers to incor- 
porate when writing the white paper (Source Document 1). 

Not all green papers are equally central to a white paper. As signaled in 
white paper 2017/18:182 (Source Document 1), SOU 2017:35 (Source 
Document 2), SOU 2016:94 (Source Document 7), and SOU 2017:51 
(Source Document 8) are of special importance to the Ministry of 
Education and Research and therefore hold a specific mandate visible in 


White Paper (N86) 


| 55.896: Idiosyncratic Political Knowledge (N48) 


Political | 44.2%: Shared (Political & Expert) Knowledge (N«38) 
Knowledge 
Expert 19 0 3 1 8 


18 9 
| 1 
GP2 GP3 GP4 GPS GP6 | GP7 GPs 
(N=337) (N=230) (N=169) (N=69) | (N=234) (N=334) (N=156) 


Fig. 8.3 The political translation of scientific knowledge in the 2015/2018 
Knowledge Achievement Reform. (Note: GP = Green paper) 
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the number of shared references with the white paper. In this case, the 
process of upward translation thus consisted of two steps of selection, 
first by announcing that certain green papers would be given special 
importance and second by being selective in the use of expert knowledge 
provided by the green papers used. 

Looking into the content of the three green papers mostly drawn upon 
by the Ministry of Education and Research, SOU 2017:35 (Source 
Document 2) and SOU 2017:51 (Source Document 8) both provide a 
wide array of expert knowledge suitable for an incremental reform that 
builds on and revises previous reforms. It is perhaps more noteworthy 
that the government also uses quite a lot of expert knowledge from SOU 
2016:94 on student absenteeism (Source Document 7). In addition to 
those three green papers mentioned as especially central to the white 
paper the Ministry of Education and Research also drew on expert 
knowledge from the green paper SOU 2016:59 On Good Grounds—An 
Action Guarantee for Reading, Writing and Math (Source Document 6). 
Launched as a result of the Swedish decline in PISA, this one-person 
commission led by an associate professor in education from Stockholm 
University focuses on developing methods for early interventions to 
improve children’s basic abilities in reading, writing, and math. The 
report uses Finland as a reference society and discusses how the early 
interventions that the commission found to be characteristic of Finland 
might explain its PISA success. Members of the commission also made 
study visits to Finland to learn about the Finnish system. 


A Final Note on the Commissions in This Study 


A brief look into the commissions that have produced the SOUs included 
in this study as source documents adds to the conclusion drawn by 
Dahlstróm et al. (2019) that there has been a significant shift in the 
Swedish committee system, away from broader parliamentary commit- 
tees toward one-person commissions with groups of advisory experts. Of 
the eight committees in this study, six were one-person commissions. 
Although based on a limited number of cases, the result coincides with 
the conclusion drawn by Christensen and Holst (2017) that civil servants 
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have been replaced by academics as chairpersons for government- 
appointed commissions. Out of the eight commissions in this study, five 
of them were led by academics employed at universities or persons hold- 
ing a doctoral degree when handing over their reports to the Minister of 
Education and Research. The most influential commission for the 
2015/2018 Knowledge Achievement Reform was the group behind GP 2 
(Source Document 2), first led by civil servant Anna Ekstróm as chair 
before she was replaced by Professor Jan-Eric Gustafsson when she was 
appointed Minister of Education and Research. 


Conclusions 


In this conclusion, we will summarize the main findings of the bibliomet- 
ric network analyses of the Swedish school reform in relation to the three 
research questions guiding the analyses. The first question concerned 
whose knowledge was used to legitimate the reform. Here, three charac- 
teristics stand out as most significant for the Swedish reform. The first is 
the high percentage of domestic, mainly governmental references. This 
finding shows the possibility for national politics to uphold a high level 
of self-referentiality even when the national political agenda to a large 
extent is dictated by international organizations such as the OECD. The 
bureaucratic machinery seems to act on its own institutional logic, rela- 
tively independent from external influence in the Swedish reform. The 
use of externalization as a way to legitimate the reform differs signifi- 
cantly between the different green papers and seems to relate to the topic. 
Some topics seem to be perceived as more national than others, which is 
illustrated through the structure of the reference list. In addition, the 
chair of a specific commission also has an important role in determining 
what knowledge is used. In the Swedish reform, it is interesting to notice 
that the chair of the most central green paper (GP 2) also stands out in 
the material as the most cited academic researcher. As researchers are 
increasingly replacing civil servants as chairs of government-appointed 
commissions in both Norway and Sweden (Christensen & Holst, 2017; 
Dahlstróm et al., 2019), what Steiner-Khamsi et al. (2020) described as 


an "expertisation of commissions" seems to create a new quasi-scientific 
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policy space between science and politics for researchers to marketize 
their own research. The results of this study thus harmonize with previ- 
ous research showing that context matters for what works in the local 
arena (e.g., Anderson-Levitt, 2003; Steiner-Khamsi, 2013). As shown by 
the results of this study, context itself has to be understood as a stratified 
concept, where aspects affecting what evidence is of the most worth and 
available for politicians to use in legitimating educational reforms are 
decided by many different interconnected, sometimes contradictory, fac- 
tors. These factors range from how the bureaucratic system functions to 
which people occupy central positions, such as being chair in a 
commission. 

The second question addressed the issue of what types of knowledge 
sources were used in legitimating the Swedish school reform. As hinted 
above, almost 6096 of the references were governmental, which is almost 
twice as much as the other Nordic countries. Another distinct feature is 
the low number of academic references, similar to Finland but much less 
than Norway and Denmark. Of the eight source documents, four do not 
have any references to academic research, which is remarkably low in a 
context emphasizing the importance of evidence-based policymaking. 
When referring to academic research, both the most important references 
are systematic reviews presenting synthesized knowledge. As the produc- 
tion of scientific knowledge increases, the role of intermediaries becomes 
equally important. These intermediaries are people, networks, or organi- 
zations whose main task is to collect, select, interpret, and present aca- 
demic knowledge for political and practical use. During the same period 
as the evolution of the 2015/2018 Knowledge Achievement Reform, 
there was a broad political consensus in Sweden on the importance of 
investing in useful knowledge. In 2015, the Swedish government even 
launched a new governmental institution called the Swedish Institute for 
Educational Research with the explicit directive to systematically collect 
and distribute research contributing to more evidence-based and effective 
teaching in Swedish schools to improve performance on national and 
international assessments. In a time of an overabundance of academic 
knowledge, the role of people and organizations operating as intermedi- 
aries thus becomes increasingly important as they provide academic 
research in a simplified and accessible way (Lubienski, 2019). However, 
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when politicians start to promote such research financially, there is a 
long-term risk that academic research could lose some of its critical stance 
in order to be perceived as relevant to policy and practice. 

The third and final question addressed the issue of what happens when 
expert knowledge is translated upward into political knowledge. The 
results show that, although the white paper drew on expert knowledge 
produced by the green papers, more than half of the references in the 
white paper were exclusive to that document. Expert knowledge is thus 
used very selectively at the political level, and politicians clearly import 
references of their own as well. Commissions offer a variety of evidence 
from which the Ministry of Education and Research can choose to 
include in the white paper. While these results confirm the idea of an 
overproduction of evidence (Lubienski, 2019; Steiner-Khamsi et al., 
2020) where much expert knowledge is lost in the upward translation, it 
is reasonable to think that some expert knowledge is imported in a pro- 
cessed form as in-text references and therefore does not turn up in the 
reference list. 

In summary, the results of this study show the importance of under- 
standing context as a complex and dynamic concept where contextual 
mechanisms at different levels affect what knowledge is used in legitimat- 
ing national educational reforms. 


Note 


1. For a more elaborated discussion on the method used in this study, see 
Chap. 3 in this volume by Oren Pizmony-Levy and Chanwoong Baek. 
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Evidence-Based Policymaking in Nordic 
Countries: Different Settings, Different 
Practices? 


Chanwoong Baek, Dijana Tiplic, and Íris Santos 


In recent years, governments have greatly emphasized evidence-based 
policymaking. By referring to evidence, policymakers attempt to ratio- 
nalize and scientificate their political claims when formulating and imple- 
menting education policy and practice (Steiner-Khamsi et al., 2020). As 
a result, references have become an essential part of the institutionalized 
practice of policymaking. It is now expected that political claims are sup- 
ported by references to evidence. References indicate that policy state- 
ments and claims are not personal or political but scientific and technical. 

Although national policymakers across the world commonly argue 
that their policy decisions are evidence based, there is little understanding 
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of how they reach such decisions. How does a country practice evidence- 
based policymaking? Specifically, what are the mechanisms deployed to 
legitimize policy proposals and new legislation? What are the common- 
alities and differences across countries in terms of what they use as “evi- 
dence"? Despite the prevalent use of references in policy documents as 
evidence, the nature of these references has been underexplored. 

In this chapter, we investigate the above questions, focusing on the 
most recent school reforms in five Nordic countries: Denmark, Finland, 
Iceland, Norway, and Sweden. There are two reasons to focus on the 
Nordic countries: first, because of the geographical proximity and strong 
historical ties among these five countries, their education models and 
policy systems are often commonly perceived to be similar; second, 
despite the similarities, there are contextual differences across these five 
countries. Thus, we discuss the policy process for an education reform in 
each Nordic country and examine how the existing mechanisms and sys- 
tems may lead to different practices of evidence-based policymaking. 


Theoretical Framework 


The practice of evidence-based policymaking in each country differs by 
(1) institutionalized forms of policymaking system, (2) degree of self- 
referentiality, and (3) type of reform. Each country has various forms of 
institutionalized policy advisory systems that help the state authorize, 
validate, or legitimize its policy decisions. These arrangements connect 
the systems of politics and science by bridging policymakers and experts. 
Interestingly, as these two systems become more coupled, science has 
faced the crisis of legitimacy. The public started questioning legitimacy 
and credibility of science after witnessing how each political assertion 
could be supported by different scientific expertise and experts (Eyal, 
2019; Maasen & Weingart, 2005). 

Eyal (2019) introduced four strategies that the state adopts to respond 
to this legitimacy crisis: objectivity, inclusion, exclusion, and outsourc- 
ing. These strategies could be categorized by the ways that the state deals 
with the problem of trust (trust in transparent, objective, public proce- 
dures vs. trust in trained judgment of experts) and the problem of 


9 Evidence-Based Policymaking in Nordic Countries: Different... 255 


extension (technocratic vs. participatory). Inclusion and outsourcing 
approaches are similar in that they both strive for participatory decision 
making; however, they differ in that the former places emphasis on open 
and public procedures, inviting various groups of stakeholders into the 
process, whereas the latter contracts out evidence production to external 
groups. By contrast, both exclusion and objectivity strategies take a tech- 
nocratic approach. The exclusion strategy establishes various gatekeeping 
mechanisms to generate an artificial scarcity of expertise, making it dif- 
ficult for one to become an expert or create organizations in charge of 
setting regulations and managing credentials. The objectivity strategy 
excludes any potential involvement of human judgments and highlights 
quantitative and objective measures. 

This typology of different strategies could be applied to examine the 
institutionalized process of evidence-based policymaking in the Nordic 
countries. Here, we hypothesize that there may be differences across the 
Nordic countries in terms of how policymakers produce and utilize evi- 
dence to claim the legitimacy of their policy proposals and recommenda- 
tions. For example, policymakers can seek evidence within the bureaucracy, 
which is closer to the exclusion strategy, or can outsource evidence pro- 
duction outside the bureaucracy, which is more related to the outsourc- 
ing strategy. Nevertheless, it is important to note that because the 
strategies are not mutually exclusive, the institutionalized policy advisory 
systems may simultaneously adopt multiple strategies. 

The characteristics of the institutionalized process are also closely 
related to the degree of “self-referentiality” in the system. The theory of 
self-referentiality states that sociological systems tend to make internal 
references and “externalize”? when they cannot address the problem 
through communication within systems (Luhmann, 1990). Thus, we 
hypothesize that a policy system with a higher degree of self-referentiality 
may have a more exclusive evidence-based policymaking process than a 
system with a lesser degree of self-referentiality. Additionally, a self- 
referential policy system might make frequent references to government 
regulations and previous decrees rather than the knowledge produced in 
external systems. 

In policymaking, references play instrumental and legitimizing func- 
tions. Policy actors could make a reference to a particular body of 
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knowledge that helps address existing policy problems or signal legiti- 
macy. A reform entailing more controversial ideas often needs references 
to first justify the problematization and then authorize and validate the 
solution (Baek et al., 2018; Howlett & Ramesh, 2003). Thus, fundamen- 
tal reforms that call for substantial changes are more likely to use refer- 
ences for their legitimizing functions. In addition, there is a greater need 
for international references in fundamental or controversial reforms 
because “externalization” helps generate crisis talk and build coalitions 
among political entities (Steiner-Khamsi, 2004). Therefore, we hypothe- 
size that fundamental or controversial reforms would have a greater num- 
ber of references in the policy documents and make more frequent 
references to international sources. 


The Cases, Data, and Methods 


The comparability of the policy process has guided our case selection, 
data, and analytic methods. As illustrated in previous chapters in greater 
detail, we focus on the most recent school reforms in each Nordic country: 


* Denmark: the 2013 Public School Reform 

° Finland: the 2014 National Core Curriculum for Basic Education 

* Iceland: Reform of 2014/2018, the renewal of the Icelandic National 
Curriculum Guide for Compulsory Schools with Subjects Areas 

* Norway: Reform of 2016/2020, the renewal of the Knowledge 
Promotion Reform 

* Sweden: Reform of 2015/2018, the renewal entitled “A Gathering for 
School—National Strategy for Knowledge and Equivalence" 


These five reforms were initiated around the same time: between 2013 
and 2016. Some reforms had the characteristics of an incremental reform 
aiming to improve previous reforms implemented relatively recently. The 
Reform of 2014/2018 in Iceland builds on the comprehensive Education 
Act for all school levels in 2008. It laid out two main goals: (1) increasing 
compulsory school pupils attainment in reading standards and (2) 
improving upper secondary students on-time graduation rates. The 
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Reform of 2016/2020 in Norway is the renewal of the Knowledge 
Promotion Reform implemented in 2006. It was designed to refine the 
Knowledge Promotion Reform of 2006, particularly focusing on the 
domains of curriculum and quality monitoring. Furthermore, the Reform 
of 2015/2018 in Sweden is the renewal of a series of fundamental reforms 
in 2011, including a new education law and new national curriculum for 
compulsory schooling. Interestingly, this renewal was heavily influenced 
by the recommendations made in the OECD study “Improving Schools 
in Sweden” (OECD, 2015). Chapter 10 explains the incremental aspects 
of the reforms of Norway and Sweden in detail. Finally, the new National 
Core Curriculum for Basic Education of 2014 in Finland is the most 
recent renewal of the national core curriculum. The new core curriculum 
did not call for major changes but instead focused on providing more 
guidance on pedagogy, emphasizing the proactive role of schools in build- 
ing a future-oriented school culture (Vitikka et al., 2015, p. 84). 

By contrast, the 2013 public school reform in Denmark required more 
fundamental changes in conceptualization and structure. The 2013 pub- 
lic school reform was one of the most recent major reforms developed in 
response to Danish students’ mediocre performance in international 
large-scale assessments (ILSAs), introducing many controversial new 
changes. In particular, a proposal to extend school hours led to much 
heated debate among stakeholders regarding its scientific basis. 

In an era of evidence-based policymaking when these five reforms are 
placed in their policy contexts in terms of national policy process and 
types of reforms (incremental vs. fundamental), questions arise regarding 
the similarities and differences across the countries in relation to what 
knowledge each government drew on to inform their policy decisions. In 
this chapter, we examine a set of key policy documents prepared for the 
selected reforms in each country, along with their references. Regardless 
of whether the author(s) made a reference to support or reject an asser- 
tion, the act of referencing is intended to provide legitimacy and credibil- 
ity to the information, claim, or evidence presented in the document. 
‘Thus, by examining the references used in policy documents, we not only 
explore what kinds of knowledge are used to inform or justify education 
reforms, but we also speculate why particular references are utilized more 
frequently than others. 
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The key policy documents were identified by the research team to best 
reflect the policy mechanisms and institutions in each country. These 
documents were put forward by a group of experts appointed or funded 
by the government or the government ministry responsible for education 
policy and practice in each country, respectively (Ministry of Children 
and Education in Denmark; Ministry of Education and Culture in 
Finland; Ministry of Education, Science and Culture in Iceland; Ministry 
of Education and Research in Norway and Sweden).! In total, there were 
five source documents in Denmark, ten in Finland, four in Iceland, ten 
in Norway, and nine in Sweden (please see Chap. 3 and previous national 
chapters for the full list of these documents). The final analytic database 
includes 5443 separate data entries of references extracted from the source 
documents. We examine these references by paying special attention to 
the type of references and location of publication across the five Nordic 
countries. We used STATA 14.2 for the descriptive and inferential statis- 
tical analyses. 


Education Policymaking Process 
in Nordic Countries 


Often, the Nordic countries have been externally and internally seen as a 
coherent group, with similar public policies representing the Nordic 
model of welfare states (Esping-Andersen, 1996; Hallsén & Nordin, 
2020). In relation to education policy, the “Nordic way” of making pol- 
icy in these five countries is frequently used as a benchmark that goes well 
beyond its geographic limits (Dovemark et al., 2018; Ringarp, 2016; 
Ringarp & Rothland, 2010). However, one may ask the following: are 
Nordic countries as similar as they are commonly perceived? 

Research has shown that despite the existence of some similarities, the 
differences among the Nordic countries are evident in terms of gover- 
nance, policy mechanisms, and institutions (see e.g., Arnesen & Lundahl, 
2006; Dovemark et al., 2018). For instance, although all Nordic coun- 
tries are highly decentralized in their comprehensive education services, 
with regional and local authorities being the main providers and owning 
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a great level of autonomy in the management of structures, teaching, and 
even curriculum, the role of private schools and the levels of school choice 
vary significantly among the Nordic countries. Indeed, while in Sweden, 
Denmark, and Iceland there is a greater number of schools to choose 
from, in Finland and Norway, school choice is rather limited (Dovemark 
et al., 2018). Moreover, although the basic governance model for non- 
public actors within primary and lower secondary education in Denmark, 
Norway, and Sweden appears to be collaborative, different policies and 
legal frameworks are applied to the mix of public, nonprofit, and for- 
profit services within education in each country (Segaard & Saglie, 2017). 
For example, the extent of private nonprofit providers is relatively large in 
Denmark, while for-profit service providers play a more significant role 
in Sweden. In Finland, although there are a few nonprofit private provid- 
ers, for-profit service providers are practically nonexistent (Lundahl, 
2017; Dovemark et al., 2018). In Norway, the roles of both of these types 
of private actors are comparatively modest (Segaard & Saglie, 2017). 
When it comes to the local governing of Nordic schools, two trends pre- 
vail: first, the degree of municipal control is much lower when it comes 
to private schools than public schools; and second, there is a higher degree 
of collaboration between municipalities and public schools than non- 
profit schools (Thøgersen, 2017). Compared with Denmark and Norway, 
Sweden is the most marketized of the three Nordic countries and has the 
largest degree of control and regulation over private schools 
(Thøgersen, 2017). 

Specifically regarding the institutionalized policy development pro- 
cess, policymaking in Nordic countries has often been perceived as tech- 
nocratic in that policy actors seek to identify the most effective solutions 
to address policy problems through scientific or technical knowledge 
(Arter, 2008; Christensen & Hesstvedt, 2019; Heclo, 1974). In particu- 
lar, Nordic countries have a long tradition of developing policies based 
on the ideas and recommendations presented in Green Papers, which are 
written by policy advisory commissions. Furthermore, Nordic countries 
have been commonly characterized as societal corporatist systems in 
which various interest groups work closely with bureaucracies. Across the 
five countries, the involvement of different stakeholders in the policy 
process is noticeable. In addition to interest group participation in 
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advisory commissions, public hearings and stakeholder consultations 
before formulating legislation to be passed in parliament are institution- 
alized processes of evidence-based policymaking found in most Nordic 
countries. 

Nevertheless, there are also national variations in their institutional- 
ized policy process. In Norway and Sweden, for example, Green Papers 
(NOUs/SOUs) are prepared at the beginning stage of policy develop- 
ment by an ad hoc commission appointed by the government when the 
government is seeking to collect policy ideas and recommendations. 
Furthermore, Green Papers stimulate debate on a particular policy issue 
among various stakeholders, organizations, and the public. Based on the 
collected knowledge and suggestions, the Ministry of Education and 
Research issues White Papers that outline the policy proposal. Although 
Green Papers may not necessarily lead to policy formulation or change, 
White Papers signify the governments intention to pass the policy. 

In Denmark, on occasion, Green Papers are released to explore policy 
issues and initiate the reform process. In the case of the 2013 public 
school reform, there were no Green Papers specifically prepared for the 
policy formulation. Instead, the reform proposal, Gor en god skole bedre— 
et fagligt loft af folkeskolen [Make a good school better—improving the 
academic level of the public school], made references to four documents 
that were authored by the government or institutions funded by the gov- 
ernment. Two of the references were authored by the Agency for the 
School Council, an independent body that consists of representatives 
from various interest groups. The other two were authored by the insti- 
tute sector, one by Danmarks Evalueringsinstitut [che Danish Evaluation 
Institute (EVA)] and the other by Socialforskningsinstituttet [the Danish 
National Center for Social Research (SFT)]. 

Although Finland also has a similar policy development process when 
it comes to Green Papers preceding White Papers for major legislative 
changes, neither White Papers nor Green Papers are necessarily required 
for curriculum reforms. For the reform that was carried out in 2014, 
however, there was the White Paper, “Future Basic Education" and sev- 
eral government-published reports were closely linked to the policy 
change. Chapter 5 explains in detail how key documents of Finland, equiv- 
alent to the White Papers and Green Papers of other countries, were 
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identified. Indeed, although the curriculum reform was meant to be 
developed in parliament and to have the participation of different stake- 
holders, it tends to be much more centralized than it appears (see Chap. 
5 for details). Ihe Green and White Papers constitute proposals by the 
Ministry of Education, which are then sent to obtain the governments 
approval. Finally, the Finnish National Agency decides on the content of 
the curriculum using the framework set by the White Paper as a basis. 

The Icelandic policy process is more decentralized. The concepts of 
White Papers and Green Papers have not been institutionalized in 
Icelandic policymaking, except for the recent White Paper published in 
2014, Hvítbók um umbatur í menntun [White Paper on Education 
Reform]. Besides this White Paper, there was an audit report prepared by 
the European Agency for Special Needs and Inclusive Education on 
behalf of the Icelandic Ministry of Education, Science, and Culture. 
Policy recommendations from this report were then adopted as education 
policy and practice, thereby functioning as a draft of the new legislation 
like a White Paper. A national policy report that provided background 
information for the Reform of 2014/2018—as a Green Paper would do 
in other countries—was the “Review of Policies to Improve the 
Effectiveness of Resource Use in Schools: Country Background Report 
Iceland." This report was prepared by the Iceland Ministry of Education, 
Science, and Culture as a part of an OECD report. Although the final 
OECD report was never produced, not only was this report cited by the 
European Agency Report, but also the comprehensive information 
included in this report often served as a reference point for policymakers 
in Iceland. 

An examination of the institutionalized policymaking process for the 
most recent education reform in each country shows that although prep- 
aration of Green Papers and White Papers for education reforms was not 
as formally institutionalized across the five countries, every country had 
its own ways of seeking expertise and information both from inside and 
outside the government. Indeed, previous literature has discussed that 
although policy advisory commissions have played a significant role in 
the Nordic corporatist policymaking context, their numbers, forms, and 
membership compositions have changed over time. Denmark, Finland, 
Norway, and Sweden had experienced a decrease in the number of 
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advisory commissions appointed for policy preparation over the past few 
decades (Ekholm & Moos, 2012; Rommetvedt, 2017). In Sweden and 
Finland, starting in the 1990s, these commissions were increasingly 
replaced by one-person commissions led by a special investigator (see 
Chap. 10; Rommetvedt, 2017). Finland even abolished the commission 
system in 2003 (Erkkilä, 2012), and the role that the commissions used 
to play is now played by broad-based working groups (Holli & Turkka, 
2019). The abolition process of commissions in Finland was complex, 
and there were various reasons behind the decision to remove the advi- 
sory commission. Since the 1990s, there had already been a growing criti- 
cism that the advisory commission system was ineffective (Holli & 
Turkka, 2019). Furthermore, the government intended to obtain control 
of the policy process by abolishing the advisory commission institution, 
which had a fair share of autonomy (Holli & Turkka, 2019). Interestingly, 
it is also suggested that the government emphasized receiving knowledge 
beyond the selected group of individuals and actively advocated for pub- 
lic hearings to stress openness and transparency in governance 
(Erkkilà, 2012). 

Under the advisory commission system, commission reports were a 
formalized part of the policy mechanism that provided information on 
debates and legislative history regarding policy issues. However, the new 
system, which includes ad hoc networks and working groups, tends to 
produce less documentary evidence regarding its deliberations (Erkkilä, 
2012). Although much information is now publicly available online, the 
available information is mostly performance and administrative manage- 
ment (Erkkilä, 2012). This also reflects the shift in modes of governance: 
the government now governs by monitoring and controlling the outputs 
(managerial accountability) instead of managing the inputs. 

Furthermore, previous literature has found changes in the composi- 
tion of advisory commissions in the five countries. Denmark, Sweden, 
and Norway have involved more academic researchers in advisory com- 
missions over time (Christensen & Hesstvedt, 2019; Christensen & 
Holst, 2017; Ekholm & Moos, 2012). In Finland, by contrast, the share 
of researchers in broad-based working groups as well as their participa- 
tion in the commissions as chairs, secretaries, and permanent experts 


have decreased (Holli & Turkka, 2019). Concurrently, interest group 


9 Evidence-Based Policymaking in Nordic Countries: Different... 263 


representation has dropped in Denmark, Finland, Norway, and Sweden 
in recent years (Binderkrantz & Christiansen, 2015; Christensen & 
Hesstvedt, 2019; Rommetvedt, 2017). 

The changes described above signal the overall decline of corporatism 
in the Nordic countries. The sources of knowledge have been diversified, 
and it has become difficult for governments to develop policies in col- 
laboration with only a few interest groups. As the number of interest 
groups has increased, the representativeness of the selected groups has 
become increasingly questionable. Furthermore, because today's policy 
interests are complex, interest groups cannot promise the support of their 
members for political exchange, which discourages governments from 
cooperating and negotiating with interest groups in the policy process 
(Rommetvedt, 2017). 

Strikingly, what is observed in Iceland differs from the rest of the 
Nordic countries. Oskarsdéttir (2018) examined the number and com- 
position of public commissions in Iceland between 1970 and 2017, find- 
ing that although the number of advisory commissions has declined over 
time in other Nordic countries, it has significantly increased in Iceland. 
The results from the study showed that Iceland now has the highest num- 
ber of preparatory corporatist commissions among the Nordic countries. 
Interestingly, when looking into the number by policy areas, the Ministry 
of Education and Culture was the agency with the highest number of 
commissions, by a wide margin. Óskarsdóttir (2018) explained that per- 
haps the reason behind why Iceland has demonstrated robust corporat- 
ism in recent years compared with other Nordic countries is because the 
state has undivided control over legislation because of its parliamentary 
majority and because a number of cohesive interest groups hold represen- 
tational monopoly. This meets the properties required for a corporatist 
exchange between the state and interest groups (Oberg et al., 2011). The 
small size of bureaucracy and limited administrative capacities are other 
potential contributing factors to Iceland’s relatively strong corporatism 
(Óskarsdóttir, 2018). 
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Reference Patterns in Nordic Policy Documents 


This study found varying reference patterns in policy documents across 
the countries in terms of (1) style and number of references, (2) type of 
references, and (3) location of publication. Each pattern reflects country- 
specific policy processes and reform contexts discussed in the previous 
sections. 


Institutionalized Practice 
of Evidence-Based Policymaking 


The frequency and style of reference could serve as an indication of the 
degree of institutionalized evidence-based policymaking practice. 
Table 9.1 shows that overall, the governments of the five countries made 
frequent references to support their policy proposals, ranging from 50 to 
264 references per policy document. This high number of references may 
not be surprising given the recent shift toward evidence-based policy- 
making. Governments are now expected to be transparent about on what 
they are basing their policy decisions. Indeed, the number of references in 
national policy documents has increased over time. For example, policy 
documents prepared for the 1997 reform in Norway seldom made refer- 
ences in their reports, and most of the references were either embedded 
in the text or listed as footnotes. However, later reforms, such as the 
Knowledge Promotion Reform of 2006 and the renewal of the Knowledge 


Table 9.1 Number of references in the policy documents 


Country AVG SD cva 
Denmark 50.20 34.95 69.61 
Finland 72.90 91.51 125.53 
Iceland 50.75 64.43 126.96 
Norway 264.50 180.26 68.15 
Sweden 179.44 116.20 64.76 
Total 143.24 145.10 101.30 


aCV (coefficient of variation) measures how the standard deviation is related to 
the mean. In this case, the higher the CV, the greater the dispersion of the num- 
ber of references across policy documents 
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Promotion Reform (2016/2020) had separate reference lists and made 
thousands of references. Even between the Knowledge Promotion Reform 
and the renewal of the Knowledge Promotion Reform (2016/2020), 
there was a significant increase in the number of references (Baek et al., 
2018; Steiner-Khamsi et al., 2020). 

Nevertheless, each country appears to engage in different levels of the 
institutionalization of reference practice. Table 9.1 shows that Norway 
made the largest number of references in a policy document on average 
(265 references per source document), followed by Sweden (179 refer- 
ences per source). By contrast, Denmark only had about 50 references 
per policy document on average. Of course, the varying length of policy 
documents may influence their average number of references because a 
longer policy document has a higher chance of having more references. 
Regardless, the low frequency of references in Danish policy documents 
is worth highlighting. For example, the reform proposal Gor en god skole 
bedre—et fagligt loft af folkeskolen did not have a separate reference section 
and had only ten references in the footnotes. 

Furthermore, Iceland has the highest variability in the number of ref- 
erences among source documents, followed by Finland. Perhaps this sug- 
gests that the reference practice in Iceland and Finland has not become as 
standardized as in other countries. Both Norway and Sweden did not 
have much variation across policy documents regarding their average 
number of references, signifying a greater level of institutionalization of 
frequent reference utilization. The patterns in the frequency of references 
in the five Nordic countries are also consistent with the institutionalized 
policy process described in the previous section in this chapter. Although 
Norway and Sweden had institutionalized the "standard model of bureau- 
cracy” through Green Papers and White Papers prior to issuing a reform 
(see Steiner-Khamsi et al., 2020; Tullock, 2005), Denmark, Finland, and 
Iceland did not mandate such a process. 

Despite the varying degrees and formats, each country has its own 
stakeholder review process where interest groups and the public can con- 
tribute their expertise to policy formulation. Furthermore, there are 
government-funded research institutes or groups of academics asked to 
conduct research for education reforms. In fact, the curriculum reform 
process in Finland seeks evidence by working closely with a variety of 
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stakeholders throughout the process. Instead of the pyramid structure of 
bureaucracy, where the government is placed at the top, experts appointed 
by the government or other stakeholders work with the government in 
the middle, and the public is at the bottom, all these actors are perceived 
to coproduce an education reform. Scholars have observed that the 
democratization of political systems has led to the participation of numer- 
ous nongovernment policy actors and organizations in policymaking 
(Maasen & Weingart, 2005). In addition, the public now monitors and 
participates in policy knowledge production, democratizing the system 
of expertise (Weingart, 2003). In other words, the democratization of 
political systems and systems of expertise has prompted governments to 
seek legitimacy and credibility beyond the traditional policy process. 


Types of References 


What constitutes evidence varies over time and is highly context-related 
(Baek et al., 2018), which translates into diverse patterns of references. 
Among the Nordic countries, not only do they vary in their styles and 
numbers of references in policy documents, but there are also variations 
in what is used as evidence. Table 9.2 shows the distribution of references 
by type of document. Overall, government-published documents are the 
most commonly cited type of reference (37.5596). However, there was a 
statistically significant difference in the types of references utilized in each 
country (X? - 823.93, p « 0.001). Although government-published docu- 
ments are the most cited type of reference in Denmark, Iceland, and 
Sweden, books and reports are the most cited type in Finland and Norway, 


Table 9.2 Distribution of references by type 


Country Report Book Journal article Government Other 
Denmark 29.87% 15.58% 16.02% 33.77% 4.76% 
Finland 11.08% 34.27% 8.86% 27.92% 17.87% 
Iceland 25.00% 3.65% 3.65% 55.21% 12.50% 
Norway 31.01% 21.19% 14.49% 26.69% 6.62% 
Sweden 11.19% 7.60% 9.15% 58.06% 14.00% 
Total 22.10% 18.06% 11.77% 37.55% 10.51% 


Note: y? = 823.93, p « 0.001 
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respectively. Furthermore, it is noteworthy that journal articles are not 
frequently cited in the Nordic countries. 


Government-Published Documents 


Our analysis reveals that the vast majority of the references in Nordic 
policy documents were produced by the government. The high presence 
of government-published knowledge in Swedish policy documents is par- 
ticularly striking considering the substantial influence of international 
organizations on Sweden's education policy agenda. In Chap. 8, Nordin 
and Wahlstróm interpret this as the government's tendency to uphold a 
great level of self-referentiality (Luhmann, 1990). The Swedish govern- 
ment favors making references to its institutionalized norms, traditions, 
and own logic. Following Sweden, policy documents in Iceland made 
many references to government-published documents. More than half 
(55.2196) of the references were prepared by the government. When 
looking into these references more closely, however, it turns out that most 
of the government-published references were the data and statistics on 
the education system produced by Hagstofa Islands [Statistics Iceland]. 
This is an interesting contrast to the government-published documents in 
other countries, which are mostly reports or proposals published by the 
respective Ministry of Education or the executive agency of the Ministry 
(e.g., Utdanningsdirektoratet [Norwegian Directorate for Education and 
Training] and Skolverket [Swedish National Agency for Education]). 
Furthermore, in Finland, only 27.92% of the references used are gov- 
ernment productions; however, a deeper analysis into the country con- 
text reveals that the influence of the government might be greater than 
that (see Chap. 5). When looking at the publishers and the authors, a 
larger amount of referenced documents are directly related to the govern- 
ment search for knowledge and evidence. The most cited publishers in 
Finland are the National Agency of Education (cited 170 times), fol- 
lowed by the Ministry of Education and Culture (57), the University of 
Jyvaskyla (55), and, in the fourth position, the University of Helsinki 
(32). Of the top four publishers, the first two are government agencies, 
and the other two are universities that hold two important and publicly 
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funded research institutes within them, Koulutuksen tutkimuslaitoksessa 
[the Finnish Institute of Educational Research (FIER)] and Helsingin yli- 
opiston Koulutuksen arviointikeskus HEA [the Center of Educational 
Assessment (CEA) in the University of Helsinki], which perform research 
on the assessment and evaluation of the Finnish education system and are 
responsible for the implementation of the Programme for International 
Student Assessment (PISA) surveys, for example. In addition, when we 
analyzed the authors of the most referenced books, reports, and academic 
research, many of the authors work for the above institutions, and often, 
their publications are the result of government-funded projects. This 
phenomenon leads to the understanding that academic and government 
productions are now often tangled, suggesting the tighter coupling 
between the systems of politics and science. 


Reports 


Although Norway had the lowest percentage of government-published 
documents as references among the five Nordic countries, it had the 
greatest percentage of reports. Indeed, reports were the second most cited 
type of reference in the Nordic countries after government-published 
documents (see Table 9.2). However, there were significant differences 
among the Nordic countries regarding the use of reports as the reference 
type. As previously stated, in Norway, reports represented the most cited 
reference type (31.01%). In Denmark, the share of reports as the refer- 
ence type was very close to that of government-published documents 
(29.87% and 33.77%, respectively). In Iceland, reports were the second 
most used reference type (25%) after government-published documents 
(55.21%). By contrast, reports were not as often used in Sweden (11.19%) 
and Finland (11.08%). 

A possible explanation for why reports are the most represented refer- 
ence type in Norway may be that the sector research in Norway has 
increased with the expansion of funding and the evaluation of educa- 
tional research programs (Zapp et al., 2018). Consistent with this specu- 
lation, Denmark, another country with a growing influence of 
state-funded independent research institutions such as EVA and SFI 
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(which is now VIVE after its merger with the Danish Institute for Local 
and Regional Government Research [KORA] in 2017), drew many of its 
references from reports (29.8796). Finally, many references to reports 
may also reflect the infrastructural and epistemological influence of inter- 
national organizations on education policymaking (see Addey, 2017; 
Sellar & Lingard, 2013). For example, in Denmark, about 2096 of the 
references to reports were produced by international organizations. In the 
following section, we discuss the role of international references in 


greater depth. 


Location of Publication 


With the increasing number of international organizations focusing on 
education, assessment, and improvement, education policymaking is no 
longer limited by the borders of each nation-state (e.g., Rizvi & Lingard, 
2010). For example, the educational reforms in many countries consider 
twenty-first-century skills for pupils learning and well-being as critically 
important at both individual and societal levels. In addition, in most 
Nordic countries, the national political, legal, and institutional frame- 
work within which education and other local welfare services are pro- 
vided has been regulated by an EU Directive (Segaard & Saglie, 2017). 
The relationships between the global and local identities working on edu- 
cation reform are increasingly tight and diverse, and they are not necces- 
sarily one-directional: although international organizations do have 
an influence on local policymaking, national policymakers utilize inter- 
national organizations and their instruments in the national policymak- 
ing process for their own agenda. One such utilization is the reference to 
the international instruments as authoritative tools to legitimize certain 
reforms (Steiner-Khamsi, 2004). Our bibliometric analysis reveals active 
utilization of international references in the Nordic policy documents, 
with Denmark having the most international references (36.3696) and 
Sweden presenting the least (18.9396). By contrast, national policy docu- 
ments made fewer references to other Nordic countries, which remain 
less than 8% of the references across the five countries (Table 9.3). 
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Table 9.3 Distribution of references by location 


Country Domestic Regional International 
Denmark 60.17% 3.46% 36.36% 
Finland 76.04% 1.63% 22.34% 
Iceland 75.00% 2.08% 22.92% 
Norway 66.83% 7.09% 26.08% 
Sweden 79.80% 1.27% 18.93% 
Total 71.94% 4.24% 23.82% 


Note: x? = 149.36, p < 0.001 


Denmark frequent utilization of international references could also 
be broken down into the five types of documents applied in the previous 
section: report, book, journal article, government-published document, 
and other. The analysis informed us that most of the international refer- 
ences in the Danish policy documents were academic literature (42.8%) 
on the topic of educational leadership (see Chap. 4). Another interesting 
finding is that many international references (16.7%) were reports pro- 
duced by the OECD. In the case of the 2013 public school reform, the 
OECD reports were referenced to identify the existing problem in the 
system and to present outsider perspectives. However, the influence of 
the OECD was not limited to providing bodies of knowledge and 
insights; it also contributed to the formulation of coalitions by ideology 
regarding educational debates, making the policy environment divided 
and antagonistic (see Chap. 11). 

Indeed, the significant influence of the OECD on educational agenda 
and policy has been discussed by many scholars (e.g., Addey, 2017; Grek, 
2009; Hansen & Rieper, 2010; Martens, 2007; Sellar & Lingard, 2013; 
Takayama, 2013), and this is not only the case in Denmark. Not only 
had each country used its PISA ranking to diagnose its education system 
or create reform pressure, but the reforms in Finland, Iceland, Norway, 
and Sweden examined in this study highlighted global education policies 
promoted by the OECD, such as competency-based education, twenty- 
first-century skills, and accountability reforms. Despite its strong subject- 
based tradition, the new core curriculum in Finland introduced and 
defined seven competence areas related to twenty-first-century skills 
(Vitikka et al., 2015). The Reform of 2014/2018 in Iceland sought to 
strengthen the competence required in society and economy of the 
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twenty-first century, which had been emphasized in the 2008 education 
policy and the National Curriculum Guides. Similarly, the Reform of 
2016/2020 in Norway built on the competency-based education intro- 
duced in the Knowledge Promotion Reform of 2006. It was also influ- 
enced by the OECD report, titled OECD Reviews of Evaluation and 
Assessment in Education: Norway 2011, which recommended assessments 
take place through learning goals and quality criteria (Baek et al., 2018; 
OECD, 2011). 

It is important to highlight that the influence of international organi- 
zations such as the OECD is not only direct but also discreet or even 
silent (see Kallo, 2009; Kauko & Varjo, 2008; Moisio, 2014; Waldow, 
2009). For example, a deeper content analysis of the Finnish White 
Papers and Green Papers found more embedded references to PISA and 
the OECD and the performative and competitive ideas expressed in their 
reports than our bibliometric analysis initially identified. Furthermore, 
although Sweden demonstrates a lower percentage of international refer- 
ences in its policy documents than that of other Nordic countries, this 
number does not exactly reflect the international effects on the develop- 
ment of the Swedish 2015/2018 Reform. Our additional analysis reveals 
that the Swedish Green Paper, “Gathering for school—national strategy 
for knowledge and equality" which had significantly shaped the 
2015/2018 reform, was specifically commissioned to review the OECD 
recommendations in "Improving Schools in Sweden" (OECD, 2015). 
Steiner-Khamsi et al. call this national adaptation of the OECD *OECD- 
reviews-in-national-disguise" (see Chap. 10). In his study of Swedish 
educational policymaking, Waldow (2009) documented a trend of "silent 
borrowing, by which international imports often remain invisible. 
Waldow (2009) explained that Swedens silence of borrowing may be 
because of its leading position in education and welfare systems after 
World War II. As a pioneer in the field, referencing external education 
systems, policies, and bodies of knowledge did not have any rationale for 
legitimization. Sweden instead relied heavily on obtaining authorization 
from scientific rationality (Ringarp & Waldow, 2016; Waldow, 2009). 
Ringarp and Waldow (2016) found that this culture shifted in the early 
2000s as Sweden lost self-confidence in educational performance because 
of its declining results in PISA; hence, it started to make more 
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international references around 2007. Despite this new orientation, the 
culture of silent borrowing may still be ingrained in the Swedish policy 
system compared with other Nordic countries, which could explain 
Sweden having the lowest number of international references. 

When it comes to referencing ones neighboring countries, Norway 
made the most regional references (see Table 9.3), suggesting its policy 
borrowing within and across the Nordic region (Steiner-Khamsi, 2004). 
Indeed, in her study of Norwegian governmental papers, Sivesind (2019) 
showed that Norwegian policymakers perceive Finland as a country of 
emulation, particularly in the areas of curriculum, quality and develop- 
ment, and student achievement (Sivesind, 2019). However, considering 
the common (mis)conception of the Nordic system as one, it is striking 
that there were not many references to regional documents. The different 
paths taken by the five Nordic countries in terms of the organization of 
the school system and its structures and management (Dovemark et al., 
2018) might explain the small usage of regional references. Alternatively, 
it is possible that the Nordic countries borrow each other's ideas, policies, 
and practices without referencing them both internationally and unin- 
tentionally (e.g., Waldow, 2009). 


Discussion and Conclusion 


In this chapter, we have examined the policy mechanisms for evidence- 
based policymaking in the five Nordic countries and the references that 
national policymakers have utilized in their policy documents to eviden- 
tiate policy ideas and recommendations. In particular, we were interested 
in whether there are similarities or differences across the five countries 
regarding what knowledge the government used to inform their policy 
decisions. The results illustrate that all five Nordic countries included in 
this study actively utilize knowledge to support and legitimate their pol- 
icy proposals; however, they do so in different ways and in different set- 
tings. The findings support most of our hypotheses that the practice of 
evidence-based policymaking varies by (1) institutionalized forms of 
policymaking system, (2) degree of self-referentiality, and (3) type 
of reform. 
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First, by comparing the policy development process of the most recent 
school reforms, we found that some countries sought evidence for policy 
proposals mainly through the policy advisory system within the bureau- 
cracy (e.g., Green Papers in Norway and Sweden), while others out- 
sourced the production of policy advice (e.g., EVA in Denmark and 
FIER in Finland). Our results also have shown that countries where evi- 
dence production is generated within the state bureaucracy (i.e., exclu- 
sion strategy) had more references than countries that produced policy 
knowledge outside the bureaucracy by outsourcing to think-tanks or 
sponsoring policy research (i.e., outsourcing strategy). This may suggest 
that in countries with an internal reference system, the bureaucracy is 
more proactive in producing and utilizing evidence or that it at least tries 
to demonstrate that its policy is evidence-based. 

National policy contexts regarding how policymakers seek policy 
knowledge (e.g., internal or external commissions, public hearings, and 
stakeholder reviews) signify each nation's political orientation and per- 
ception toward democratic and technocratic policymaking. Furthermore, 
the change in political models, such as the prosperity of corporatism in 
Iceland and the decline of corporatism in other Nordic countries, shapes 
who participates in the policy process and what their roles are. This infor- 
mation contributes to a more complete understanding of the boundaries 
of rationality and knowledge, which consequently influences the refer- 
ence utilization in each country. 

Second, our analysis of reference utilization in the five Nordic coun- 
tries showed that reference utilization depends on the extent the policy 
system is self-referential or receptive to externalization. The frequent uti- 
lization of non-government-published documents such as reports, books, 
and journal articles in Norway indicates that policymakers were open to 
external sources of knowledge beyond the system of politics. Indeed, the 
Norwegian government extensively made use of the knowledge produced 
by institute-sector organizations such as the Nordic Institute for Studies 
in Innovation, Research and Education (NIFU) and Norwegian Social 
Research (NOVA), both of which bridge the systems of politics, science, 
and practice. Norway also demonstrated a relatively high percentage of 
international references that were produced across geographical boundar- 
ies. This is interesting considering that Norway demonstrates a higher 
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level of externalization despite having an exclusive policy advisory sys- 
tem. On the contrary, Sweden, another country where the evidence- 
seeking process is centered in the state bureaucracy, demonstrated a 
greater tendency toward self-referentiality or was at least less explicit 
about its externalization. Sweden had the highest percentage of 
government-published documents and the lowest percentage of interna- 
tional references compared with the other Nordic countries. Despite this 
concentration on internally produced knowledge, many of these internal 
references turned out to be local translations of externally produced 
knowledge (see Chap. 10; Waldow, 2009). In other words, it is possible 
that externalization was disguised in the form of internal references. 
When discussing externalization and translation to understand reference 
patterns, an additional factor to consider is that there could be certain 
bodies of knowledge that had become common knowledge within the 
system and did not require any formal reference. Thus, the absence of 
references to particular knowledge does not necessarily mean the disre- 
gard of the knowledge. 

Third, although our hypothesis that a fundamental or controversial 
reform would have more references in an attempt to obtain legitimacy 
and scientific base did not hold, the results confirmed another hypothe- 
sis: a fundamental or controversial reform would utilize more interna- 
tional references than an incremental or noncontroversial reform. An 
example from our analysis could be the Danish case—a highly controver- 
sial reform—which has the most active international reference utiliza- 
tion. In-depth analyses of international references in Denmark showed 
that policy actors used international references, particularly the ones pro- 
duced by the OECD, to legitimize the need for the 2013 public school 
reform by problematizing and diagnosing the existing system (see Chaps. 
4 and 11). By contrast, for the 2016/2020 Reform in Norway, which was 
an incremental reform of the 2006 Knowledge Promotion Reform, pol- 
icy experts who served on advisory commissions shared that they often 
referred to international references to collect knowledge about a policy 
topic that had not been explored in-depth domestically (Baek, 2020). 

In conclusion, our findings show that there are similar and different 
patterns in institutionalized policy processes and reference utilization 
across the Nordic countries. These differences could be understood by a 
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combination of three factors: institutionalized policymaking system, self- 
referentiality, and type of reform. For future studies, we suggest a more 
in-depth analysis of the interplay between the three factors to better 
understand particularity of the national approaches to evidence-based 
policymaking. 


Notes 


1. Borne- og Undervisningsministeriet in Danish; Opetus- ja kulttuuriminis- 
teriö in Finnish; Mennta- og menningarmdlardduneytid in Icelandic; 
Kunnskapsdepartementet in Norwegian; Utbildningsdepartementet in 
Swedish. 

2. Swedish: Samling for skolan—Nationell strategi for kunskap och likvärdighet. 
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According to several scholars in political science, government- 
appointed advisory commissions are typically established to fulfill three 
broader purposes: expertise, accountability, and representation (e.g., 
Boswell, 2017, 2018). Governments rely on experts with insider knowl- 
edge who are sufficiently familiar with the bureaucracy to provide useful 
and realistic advice regarding complex matters. Ideally, independent 
experts—preferably academics working outside the bureaucracy—are 
needed to provide credible expertise. These independent experts are 
authorized to observe and evaluate past reforms and, by implication, to 
hold the administration accountable for its technocratic performance. 
Finally, governments must satisfy the demands of their political environ- 
ment for participation and representation in government decisions. 

In reality, however, the NOUs and SOUs have been directly affected 
by the “erosion of the corporatist model” (Lundberg, 2015) or “corporat- 
ism in decline” (Rommetvedt et al., 2012), a clear trend that political 
scientists have explored in great detail. In the research literature, the 
twentieth-century governance models of Denmark, Norway, and Finland 
have frequently been showcased as “societal corporatist” systems, in 
which interest groups exerted active influence on the government by 
direct participation in commissions and direct interactions with govern- 
ment civil servants. In effect, the rise of this powerful state apparatus 
sparked a gradual process of depoliticization, whereby power was given to 
government bureaucrats rather than elected members of parliament. This 
weakening of parliamentary power as a result of societal corporatism has 
been a recurring theme in political science and policy studies 
(Rokkan, 1966). 

The perceived attack on the strong state has triggered a series of changes 
at the heart of government. In the face of “the revival of parliaments” in 
Norway and Sweden, interest groups such as unions, business organiza- 
tions, and professional associations have sought out new channels for 
influencing political decisions. Rather than following the traditional state 
corporatist model of providing input to government-appointed advisory 
commissions, these interest groups chose to change their political arena: 
withdrawing from the NOUs and SOUs and focusing instead on lobby- 
ing politicians and elected members in the parliament directly (Lindvall 
& Rothstein, 2006; Østerud & Selle, 2006). The devolution of power 
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from government to parliament has thus impacted not only the interest 
groups’ agenda setting and communication strategies but also the com- 
position of NOUs and SOUS, their political reach, and the wider purpose 
of the "policy advisory system" (Halligan, 1995; see also Craft & 
Howlett, 2013). 

From a broader policy perspective, this shift has also impacted how 
education is governed in an era of international comparison and evidence- 
based policy planning. Over the past decade, public policy scholarship 
has documented a movement toward network governance (Ball & 
Junemann, 2012) and multi-centric policymaking (Cairney et al., 2019). 
Across all public sectors, including education, agenda setting and policy- 
making is now carried out in multiple sites and involves multiple actors. 
To complicate the policymaking process further, new actors, notably 
international actors and non-state actors, have become increasingly influ- 
ential in terms of national agenda setting and policymaking: previous 
studies have highlighted the proliferation of informal channels of consen- 
sus- and coalition-building in government (e.g., Rommetvedt et al., 
2012). Governments, including those of corporatist states, are evidently 
under constant pressure to interact and negotiate with, and to mediate 
between, a plethora of policy actors, ranging from traditional politicians 
to this new wave of interest groups. 

In our comparative bibliometric network analysis of NOUs and SOUs, 
we analyze reports from these commissions (Green Papers) in terms of 
the publications referenced to support their assertions (see Chap. 2 in this 
book). Equally important is the analysis of the official White Papers 
themselves, which considers the received knowledge on which the respec- 
tive Ministries of Education and Research draw when explaining and 
justifying reform. Finally, we investigate the relationship between the two 
types of documents (advisory versus decision-making): which references, 
and how many, that are listed in the commission reports also surface in 
the actual policy documents? We have termed this latter process—the 
knowledge transfer from Green to White Paper— political translation. 
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Green and White Papers: Important Milestones 
in the Policymaking Process 


Against the backdrop of the devolution of power from government to 
parliament in Norway or from executive to legislative in Sweden, the first 
research question may be formulated as follows: How have the respective 
Ministries of Education and Research repurposed their advisory commis- 
sions? The second research question focuses on the official use of commit- 
tee guidance on policy by the respective Ministries. Specifically, we 
examined the extent to which the commission reports (NOUs and SOUs) 
draw on a similar body of knowledge as the White Papers.' 

It is first necessary to explain the role of Green and White Papers in the 
larger policymaking process. Figure 10.1 depicts the general process of 
policy formulation in Norway and Sweden. As shown below, Green 
Papers constitute the first stage in preparing a new policy. Green Papers 
are then shared with stakeholders (interest groups, professional associa- 
tions, etc.) for review and feedback, known as “hearings.” The Ministry 
of Education and Research then prepares the White Paper based on the 
commissioned Green Papers along with the feedback received. 

In both countries, White Papers are produced by the executive body in 
the education sector, the Ministry of Education and Research. In Sweden, 
White Papers are then signed by both the Minister of Education and 
Research and the Prime Minister. In Norway, the Council of State (the 
King and the government) formally approves the White Paper. Thereafter, 
the Standing Committee on Education and Research submits its recom- 
mendations to the legislative body (parliament), which is responsible for 


Green Papers Referral Review 
(NOUs, SOUs) Process White Papers Process Resolution 
Advisory Stakeholders Ministry of Parliamentary/ Parliament 
Commissions Education and Standing 
Research Committees 


Fig. 10.1 The policymaking process in Norway and Sweden 
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making the final decision. Our focus here is on the transfer of knowledge 
from the advisory body to the decision-making authority, a process we 
call political translation. In addition, comparative analysis of the two 
policy-advisory systems enables us to distill context-specific features of 
the policymaking process in Norway and Sweden. 

Ultimately, this study aims to address the common suspicion held by 
the public when it comes to the seemingly innumerous government- 
appointed advisory commissions: Do governments merely use them for 
window dressing, or do they listen to their advice? If they do, which 
advice is taken, which is left out, and which is politically reframed, 
and why? 


Methodology: The Bibliometric Database, Case 
Studies, and Comparative Methods of Inquiry 


Certain methodological explanations are in order here. In particular, a 
more detailed description of the bibliometric database and the compara- 
tive research design may help the reader to put our findings in 
perspective. 


Sampling of Source Documents 
and Bibliometric Database 


As explained in greater detail in previous chapters, our bibliometric data- 
base was drawn from the references of White Papers and commission 
reports. We used a set of White Papers prepared for the renewal of the 
Knowledge Promotion Reform (2016/2020) (Norway) and the Swedish 
School Reform of 2015/2018 as source documents. Those commission 
reports that were explicitly mentioned in the identified White Papers 
were also added. We extracted all references from the source documents 
and entered them into the database. It is important to note that NOU 
2015:2 was cited in both White Papers in Norway; NOU 2014:7 and 
NOU 2015:8 were produced by the same commission (the Ludvigsen 
Commission); and SOU 2016:66 did not have any formal references. In 
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total, the White Papers and commission reports from Norway and 
Sweden cited 2312 documents and 1421 documents, respectively. 
Table 10.1 presents the composition of the bibliometric database. 

In this chapter, we turned our attention to the reference attributes, 
including types of document and location of publication. We employed 
the categorization used by Christensen and Holst (2017) for coding. The 
eight values for types of documents were as follows: (1) national policy 
documents; (2) national policy research; (3) national academic research; 
(4) international policy documents; (5) international policy research; (6) 
international academic research; (7) interest groups, think tanks, and so 
on; and (8) others. We then clustered these values into three location 
categories: (1) national, (2) international, and (3) others. To ensure inter- 
coder reliability, we first discussed the coding scheme extensively, carried 


Table 10.1 Bibliometric databases (Norway and Sweden) 


Country Norway Sweden 
Reform 2016/2020 Knowledge Renewal 2015/2018 Knowledge 
Reform Achievement Reform 
Source WP, n=2; GP, n=8 WP, n=1; GP n=8 
documents 
White WP (2015/16) WP (2016/17) WP (2017/18) 
Papers Renewal of the Early Intervention Gathering for School 
Knowledge and Quality in 
Promotion schools^ 
Reform? 
Green NOU 2003:16; NOU 2009:18; SOU 2008:52; SOU 
Papers NOU 2007:6; NOU 2010:7; 2013:56; SOU 2015:22; 
NOU 20147; NOU 2015:2; SOU 2016:59; SOU 
NOU 2015:2; NOU 2016:14 2016:66; SOU 2016:94; 
NOU 2015:8 SOU 2017:35; SOU 
2017:51 
References? 2312 1421 


Translation of the full title: Subjects, in-depth learning, understanding: A renewal 
of the Knowledge Promotion Reform; Norwegian: Fag—fordypning— 
forstáelse—En fornyelse av Kunnskapsloftet 

PTranslation of the full title: Eager to learn: Early intervention and quality in 
schools; Norwegian: Leerelyst—tidlig innsats og kvalitet i skolen 

‘In Swedish: Samling fór skolan 

*Number of documents cited directly in both White Papers (WP) and Commission 
Reports. Source documents are also counted as long as they are cited 
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out independent reviews, and (in case of divergence) reiterated the review 
and coding. We used UCINET 6.681 and NetDraw 2.168 for network 
data analysis and visualization and STATA 14.2 for statistical analysis. 


Units of Analysis: School Reform in Norway 
and Sweden 


A brief outline of the substance of school reforms in both countries and 
of the specific mandates of the government-appointed commissions 
allowed us to place the two types of source documents— White Papers 
and Green Papers—into their larger policy context. 


Norwegian School Reform 2016/2020 


In Chap. 7 of this book, we examined the renewal of the Knowledge 
Promotion Reform (2016/2020) in Norway. 'Ihis reform was initially 
tabled by two White Papers produced by the Ministry of Education and 
Research and enacted by parliament in 2016 and 2017. The reform was 
planned to go into effect in 2020. It is considered an incremental reform, 
which reconfirms the Knowledge Promotion Reform launched a decade 
earlier. It is therefore necessary to discuss the current reform in its histori- 
cal context. 

The Norwegian school reform known as the Knowledge Promotion 
Reform came into effect in 2006, replacing two previous reforms of pri- 
mary and lower secondary curricula as well as upper secondary educa- 
tion. It can be considered a fundamental reform (Steiner-Khamsi et al., 
2020) as it represented a shift from input-oriented to output-oriented 
policy instruments, such as measurable objectives, standardized tests, and 
data-based planning (Moller & Skedsmo, 2013). As pointed out by other 
researchers, the reform was partly motivated and legitimized by the results 
of the PISA test released in December 2001 (see Skedsmo, 2018). 
Alarmed by the PISA findings, the government concluded that the school 
system had serious weaknesses in need of immediate repair. To meet these 
challenges, the Ministry of Education and Research suggested the 
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introduction of a national testing system and improvements to the com- 
petencies of teachers, school leaders, and administrators through the 
establishment of a “culture of learning” (Karseth & Sivesind, 2010). The 
reform brought about increased decentralization, on the one hand, and 
increased accountability, on the other. 

‘The revised national curriculum targeted specific competencies for stu- 
dent learning outcomes and emphasized basic or foundational skills that 
were supposed to be integrated in all subjects and across all grades (Imsen 
& Volckmar, 2014). Furthermore, a national quality assessment system 
was introduced alongside the curriculum reform (Moller & Skedsmo, 
2013), and national testing was first implemented in 2004. For the first 
time, the test results were published and made publicly available, allow- 
ing schools to be benchmarked, ranked, and compared. Undoubtedly, 
this shift toward outcomes-based monitoring represented a radical break 
with and departure from the traditionally input-based regulation of 
Norwegian education (Helgoy & Homme, 2016). 

Seven years later, the Ministry concluded in hindsight that the 2006 
reform had been an overall success, as exemplified by rising scores in 
international large-scale student assessments, among other indicators 
(Ministry of Education and Research, 2013, p. 12). However, it also 
found that certain shortcomings in the reform had become a cause for 
public concern. First, the curriculum was seen as overloaded, leading the 
Ministry of Education and Research to suggest that priorities—in terms 
of both content knowledge and subjects—needed to be set based on evi- 
dence and formative evaluation. Second, as a result of prioritization and 
the emphasis on deep learning, the Ministry of Education and Research 
mandated that the key elements for each school subject be defined in 
greater detail. Third, acknowledging the importance of social develop- 
ment, three interdisciplinary topics were given high priority: democracy 
and citizenship, sustainable development, and public health and well- 
being. Fourth, the reform introduced remedial measures for students 
with low achievements in reading, writing, and numeracy in grades one 
to four. Among the many improvements that the 2020 reform intended 
to achieve, one more is worth mentioning: the reform reaffirmed the 
principle of “test-based accountability” (Verger & Parcerisa, 2018), 
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whereby local authorities are held accountable for student learning out- 
comes in the schools under their jurisdiction. 

Consequently, the Ministry put in motion a two-pronged incremental 
reform process to renew the existing curriculum and to develop the 
national quality system further. We have labeled the resulting 2020 school 
reform the Curriculum Renewal/Quality Monitoring Reform and, for 
easier reference in this book, merged the two as the renewal of the 
Knowledge Promotion Reform (2016/2020). This is the reform investi- 
gated in this chapter. In particular, we examine two White Papers that the 
Ministry of Education issued in 2016 and 2017, respectively, when it 
announced the 2020 reform: the Renewal of the Norwegian Knowledge 
Promotion Reform (Ministry of Education and Research, 2016)? and the 
Earl Intervention and Quality Monitoring Reform (Ministry of 
Education and Research, 2017).? 


Swedish School Reform 2015/2018 


In the 1990s, Sweden introduced a series of neoliberal school reforms 
that featured large-scale privatization and a decentralization of decision- 
making authority from the central to the local level. Within a short period 
of time, the 290 municipalities were put in charge of overseeing compul- 
sory schooling. The 2015 Final Report of the School Commission, 
Gathering for School (SOU 2017:35), set the reform in motion, which 
came into effect in 2018 (White Paper 2017/18:182). In this book, we 
have therefore labeled the Swedish school reform the 2015/2018 
Knowledge Achievement Reform. 

Naturally, the deterioration of the strong Swedish welfare state and free 
public services has not gone unnoticed in the research literature (Englund, 
1996; Lundahl, 2007). Weaponized by weak performances in the PISA 
tests from 2003 to 2012, when Sweden reached its lowest scores, vocifer- 
ous public debates about the education crisis spread across the country 
(Nordin, 2019). In 2006, the Swedish government agreed to instate a 
commission to investigate the challenges in the education sector. Based 
on that decision, the Ministry of Education and Research appointed a 
commission with the mandate to explore the reasons for the rapid fall in 
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PISA scores. In practice, the commission was a one-person commission 
without any members, chaired by the Assistant Under-Secretary of the 
Ministry of Education and Research, Leif Davidsson. The commission 
report, Clear Goals and Knowledge Demands in Elementary School (SOU 
2007:28), asserted that the compulsory school curriculum was too vague 
and ideological and criticized the decentralization reform for its detri- 
mental effects on equal opportunities and equality. According to the 
report, the neoliberal reforms of the 1990s and the turn of the century 
had benefitted only a precious few municipalities, leaving the overwhelm- 
ing majority struggling to offer quality education. 

Following the recommendations of the commission report, the Swedish 
government launched a second series of fundamental reforms in 2011, 
the most prominent features of which were a new education law and a 
new national curriculum for compulsory schooling that paid great atten- 
tion to learning outcomes and the reform of teacher education. A more 
refined grading system was introduced, and test-based accountability was 
actively pursued: students had to take a higher number of national, stan- 
dardized tests, which allowed the Ministry of Education and Research to 
resume oversight over the quality of education. This outcomes-based 
reorientation was not out of the ordinary per se, but in Sweden it enabled 
the Ministry of Education and Research to reclaim central control over a 
school system that had been decentralized only a decade earlier. 

However, the introduction of clearly defined standards, unambiguous 
accountability measures, and massive investments into the Swedish edu- 
cation sector did not yield the expected results. In 2012, only a year after 
the launch of the extensive reform package, Sweden reached its lowest 
PISA scores ever. Humiliated, the government turned to the OECD, 
seeking help with analyzing the problems of the Swedish school system as 
well as recommendations on how to fix them (see Grek, 2019; Pettersson 
et al., 2017). The resulting OECD findings (2015) were presented in 
Improving Schools in Sweden, a study that proposed comprehensive 
reforms in three priority areas: (a) to establish conditions that promote 
quality with equality across Swedish schools; (b) to build capacity for 
teaching and learning through a long-term HR strategy; and (c) to 
strengthen steering of policy and accountability with a focus on improve- 
ment. As a follow-up, the Swedish government appointed a commission 
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with the explicit mandate to review the OECD proposals for educational 
reform. The commission review of the OECD analyses and recommen- 
dations was published in the SOU report Gathering for School: A National 
Strategy for Quality and Equivalence. Final Report of the 2015 School 
Commission (2017:35).’ It would appear that the Ministry of Education 
and Research took this particular SOU report to heart, as it became the 
foundation for the 2015/2018 school reform in Sweden. Indeed, even 
the name of the subsequent White Paper, Gathering for School, was taken 
from its Green Paper precedent. 


Comparative Design 


Norway and Sweden are commensurable in terms of their institutional- 
ization of the policymaking process. The practice of instating advisory 
councils, for example, dates back to the sixteenth century (Lundberg, 
2015). The system has naturally changed over time and, following the 
independence of Norway and Sweden from Denmark, bifurcated in two 
different directions. Today, however, the two systems have remained suf- 
ficiently similar to be considered comparable when it comes to the poli- 
cymaking process. 

As Fig. 10.2 illustrates, the research design enabled us to carry out 
three types of comparison: (1) of policy-advisory systems; (2) of policy- 
decision systems; and (3) of knowledge transferred from the former to 


| Green Papers White Papers 
Produced by Policy Advisory Commissions Produced by Ministry of Education and Research 
(NOUs & SOUs) (MER in Norway & Sweden) 


— ~ 


3a. Shared Knowled e a] 
Norway e 


(Political translation in Norway) 
1. Comparison of 3 3a & 3b 3 2. Comparison of 
Policy-Advisory Systems TT eea] ? e Policy-Decision Systems 
Sedan 3b. Shared Knowledge 
(Political translationin Sweden) 


Fig. 10.2 Overview of the comparative research design 
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the latter. We have coined the term political translation to denote this 
particular transfer process; this third type of comparison entails identifi- 
cation of shared knowledge between the commission reports and White 
Papers in Norway (3a) and Sweden (3b), as well as a comparison of these 
two patterns of political translation (3c). 

Specifically, the three types of comparison draw on the following data: 

Type 1—Comparison of the two policy-advisory systems. The data- 
base consists of eight commission reports from Norway (NOUs) and 
eight commission reports from Sweden (SOUs). How do these two sets 
of reports differ in terms of their references? In particular, are there differ- 
ences in terms of their national/international orientation and the type of 
documents that they reference? 

Type 2—Comparison of the two policy-decision systems. The Ministry 
of Education and Research of Norway outlined the renewal of the 
Knowledge Promotion Reform (2016/2020) in two White Papers, which 
together contain a total of 294 references. For the 2015/2018 Knowledge 
Achievement Reform in Sweden, the Ministry of Education and Research 
issued one White Paper that includes 86 references. For a comparative 
bibliometric analysis, the key questions are as follows: On what knowl- 
edge sources do the two ministries draw? Is there a preference for certain 
types of texts? Do the two ministries differ in their selection of refer- 
enced texts? 

Type 3—Comparison of commission reports and White Papers (polit- 
ical translation). A comparison of the two types of documents with dif- 
ferent functions (advisory versus decision-making) yields interesting 
insights in terms of political translation. Methodologically, we compared 
the ratio of shared knowledge/references as a percentage of all references 
listed in the commission reports of each respective country. In other 
words, what percentage of references listed in the commission reports 
also appear in the reference section of the White Papers? The ratio of 
shared references is interpreted as an indication of the political transla- 
tion process. After completing the analysis for each of the political sys- 
tems (3a and 3b), we then compared the two political translation processes 
in Norway and Sweden (3c). 
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Research Findings 
Comparison of the Policy-Advisory Systems 


A juxtaposition of the references listed in the respective commission 
reports highlights two key similarities and two key differences between 
the policy-advisory systems of the two countries that are worth exploring 
in greater detail. 

In terms of shared characteristics, the commissions in both countries 
substantiate their reviews and their recommendations with a large num- 
ber of references. For Norway, the number of references listed in NOUs 
ranges from 146 to 703 references, with an average of 292 references per 
report. This practice of excessive referencing may also be found in the 
Swedish policy-advisory system: the SOUs show a maximum number of 
references of 337 and an overall average of 191.5 The pressure to make 
knowledge sources transparent and to provide "evidence" for the com- 
missions assertions is clearly discernible. A second commonality is the 
national orientation of the commissions in terms of their reference litera- 
ture: approximately two-thirds of the references listed in the commission 
reports for both countries were published domestically. 

Both of these phenomena deserve theorizing. The first confirms the 
belief in knowledge-based or evidence-based policy advice. It is expected 
of modern-day commissions that they read and reference relevant texts 
excessively and ostentatiously. Second, the commissions in both coun- 
tries tend to cite national authors, that is, either themselves or authors 
known to them. The large proportion of national references may also 
reflect the practice of commissions to reflect and report on contemporary 
topical debates and controversies (Sweden: low learning outcomes and 
student absenteeism; Norway: drop-outs and concerns about students' 
psychosocial environment). 

There is, however, a statistically significant difference (p « 0.05) 
between the types of references listed in the commission reports of the 
two countries. In Norway, 41.2% of all references in the eight examined 
commission reports (867 out of 2106 references) are academic research. 
In stark contrast, academic studies make up only 19.796 of the references 
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Table 10.2 Types of references in commission reports and White Papers 


Norway Sweden 
Type GP WP GP WP 
National policy documents 443 47 448 61 
21.04% 15.99% 34.81% 70.93% 
National policy research 481 112 410 13 
22.84% 38.10% 29.86% 15.12% 
National academic research 403 24 92 0 
19.14% 8.16% 6.70% 0 
International policy documents 85 5 38 4 
4.04% 1.70% 2.77% 4.65% 
International policy research 101 44 57 8 
4.80% 14.97% 4.15% 9.30% 
International academic research 464 53 178 0 
22.03% 18.03% 12.96% 0 
Interest groups, think tanks, andsoon 35 1 71 0 
1.66% 0.34% 5.17% 0 
Others 94 8 49 0 
4.46% 2.72% 3.57% 0 
Total 2106 294 1373 86 


100% 100% 100% 100% 


in the eight examined SOUs from Sweden (270 out of 1373 references). 
Instead, the Swedish policy-advisory system relies heavily on national 
policy documents (34.81%) and national policy research (29.86%) to 
support commission reviews and recommendations (see Table 10.2). 
The academization of Norway’s policy-advisory system is a well-studied 
phenomenon (Christensen & Holst, 2017). In our bibliometric network 
analysis of the 2006 Knowledge Promotion Reform in Norway (Steiner- 
Khamsi et al, 2020), we cursorily examined the composition of 
government-appointed commissions over time and found that the num- 
ber of academics appointed had increased dramatically over the previous 
twenty years. Our observations concur with those of a larger study on 
expert commissions (Christensen & Holst, 2017; Christensen & 
Hesstvedt, 2019), which considered the composition of advisory com- 
missions across various ministries over a period of close to fifty years. 
What they found is striking: the representation of interest groups in edu- 
cation sector commissions has dropped sharply and is now half of what it 
was four decades ago. During the same time span, the proportion of 
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academics on education-related NOUS has seen a nearly fourfold increase: 
from 9% in the 1970s to 35% in the 2000s. The dramatic upsurge of 
researchers came at the expense not only of the interest groups but also of 
civil/public servants: whereas forty years ago, more than half of NOU 
representatives were government officials, they now constitute only one- 
third of commission members. 

Today, the two largest groups represented in the NOUS are researchers 
and government officials, which together account for more than 9096 of 
members. In contrast, interest group representation stands at less than 
10. The explosive growth of researcher representation in NOUS has led 
Christensen and Hesstvedt (2019) to suggest that further investigations 
are necessary to understand the "expertization" of Norwegian advisory 
commissions in greater detail. The breakdown of commissions by chair- 
persons and secretariats portrays a similar picture. As Christensen and 
Holst (2017) have asserted, academics have replaced civil servants as 
chairpersons in the majority of commissions since the year 2000. In other 
words, academics have become the public face of the commissions. 

By contrast, the academization of the Swedish advisory commissions is 
conspicuous by its absence. Our bibliometric study demonstrates that 
only 19.7% of the references consist of academic research. This is surpris- 
ing given the heavy representation of academics as chairs of government- 
appointed commissions: five out of the eight commissions included in 
this study were led by academics who are either employed at universities 
or have completed their doctoral degree. Two were led by politicians and 
one by a civil servant. On the one hand, then, academics are visible and 
influential as chairs of Swedish advisory commissions, as seen in Norway 
above. A case in point is the change in chairpersonship for the commis- 
sion that produced the influential Gathering for School Green Paper in 
2017. When the commission was established in April 2015, Anna 
Ekstróm, an influential civil servant and Director-General of the Swedish 
National Agency for Education (Skolverket), was appointed as the chair, 
but when she left the commission in September 2016 to assume the posi- 
tion of Minister of Education and Research, she was replaced by Jan-Eric 
Gustafsson, a professor of education at the University of Gothenburg. 

On the other hand, a distinct feature of the Swedish policy-advisory 
system is the move from larger advisory to one-person commissions 
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known as Special Investigator Commissions. Whereas the former advi- 
sory commissions served as a representative and participatory body of 
various stakeholders (mirroring the composition of parliament), each 
Special Investigator Commission is led by an individual considered 
appropriate by the Ministry of Education and Research. The appointed 
individual is often a politician, civil servant, or academic with a group of 
associated experts at his or her disposal for consultation. This gradual 
shift from advisory commissions to Special Investigator Commissions 
that began more than two decades ago, therefore, has meant a reduction 
in terms of representativeness and of parliamentary and societal legiti- 
macy as a result. 

In the Green Paper SOU 1999:121, Hermansson and colleagues 
examined 509 commissions and found that the share of Special 
Investigator Commissions had doubled from 3096 in 1960 to 6096 in 
1995. This development has been confirmed in a more recent analysis by 
Dahlstróm et al. (2019), who demonstrated that in 2016, as many as 
9096 of all government-appointed commissions in Sweden were led by 
Special Investigators. The removal of politicians and interest group repre- 
sentatives from these commissions tipped the balance between represen- 
tativeness and efficiency: drastically shortening the investigation period 
from a few years, in extreme cases, to only a few months or even weeks 
(Dahlström et al., 2020), while downsizing, expertizing, and depoliticiz- 
ing the commissions themselves (see Petersson, 2016). Indeed, the dimin- 
ished political influence of government-appointed commissions is well 
captured by Lundbergs verdict: “injured but not yet dead” (2015). The 
pattern is also repeated in the commissions that advised the Ministry of 
Education and Research on the most recent reform: of the eight SOUs 
identified as relevant to the 2015/2018 Knowledge Achievement Reform, 
six were produced by a Special Investigator Commission. Against this 
general pattern, it is worth noting that the Gathering for School Green 
Paper (SOU 2017:35) chaired by Professor Jan-Eric Gustafsson, and cen- 
tral to the 2015/2018 Knowledge Achievement Reform, is one of the two 
SOUs produced by larger advisory commissions. 

Another historical characteristic of Swedish advisory commissions has 
been a considerable degree of autonomy in relation to central govern- 
ment (Tragardh, 2007). However, as underlined by Petersson (2016), the 
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shift away from broadly representative and politically powerful commis- 
sions to smaller Special Investigation Commissions has led to a tighten- 
ing of control by the ministries and a subsequent reduction of autonomy. 
Significantly, the twin processes of depoliticization and bureaucratization 
of the commissions have served to increase governmental power over the 
policy-advisory process. This is also evidenced by the rise, from 1990 to 
2014, in the number of commission directives (kommitédirektiv), the 
written instruction to the appointed commission stating its missions, 
mandate and timeframe, and the most important tool with which the 
government can control commissions (Dahlstróm et al., 2020). Crucially, 
the government can also adjust or replace these directives over time as a 
way to exert control within the process. 

Clearly, then, both countries have chosen different pathways to cope 
with the corporatist dilemma of the 1980s and 1990s. In Norway, inter- 
est groups reduced their presence in the commissions and instead sought 
more effective means of exerting political influence. Together with the 
devolution of power from government to parliament, this led to a depo- 
liticization of the commissions. Longitudinal studies (Christensen & 
Holst, 2017; Christensen & Hesstvedt, 2019) have suggested that the 
empty seats left behind by the political interest groups were filled by aca- 
demics and that political representation was jettisoned in favor of scien- 
tific expertise. 


Comparison of the Policy-Decision Systems 


The heavier reliance on research (both independent/academic and com- 
missioned/applied policy research) in the Norwegian policymaking pro- 
cess also resonates at the ministerial level. As Table 10.2 shows, of the 294 
references listed in the two White Papers on Norwegian school reform 
(WP 2015/16 and WP 2016/17), an overwhelming majority (79.2596) 
fall under either national policy research (38.1096), national academic 
research (8.16%), international policy research (14.97%), or interna- 
tional academic research (18.0396). According to the categorization sys- 
tem used by Christensen and Holst (2017), we found that four in five of 
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the references (233 out of 294) represent analytical publications, that is, 
national or international policy research or academic research. 

Conversely, in Sweden, analytical publications make up less than a 
quarter (24.42%) of the references listed in the ministerial Gathering for 
School White Paper. It is equally important to note that the Ministry of 
Education and Research has not cited a single academic publication, 
whether national or international. Ihe marked absence of academic refer- 
ences in the Swedish White Paper calls into question whether the Ministry 
has a troubled relationship with its own academic community and with 
foundational research in general. 

A second finding has been somewhat surprising, too: in Norway, the 
Ministry of Education and Research cites significandy more research- 
related publications in its White Papers than its appointed expert com- 
missions do in their reports. Specifically, the bibliographies of the White 
Papers comprise 79.25% analytical publications, compared with 68.8% 
for the NOUS (see Table 10.2). 

Both of these unexpected results require further unpacking: the first in 
light of the differentiation between “mode 1” and “mode 2" knowledge 
(Nowotny et al., 2003) and the second in light of the importance of the 
chairpersonship to the commissions work. 

First, the differentiation between foundational, academic knowledge 
(mode 1 knowledge) and applied, policy research (mode 2 knowledge) 
helps us to understand why, at first glance, the Ministry of Education and 
Research of Norway seems to be more committed to research than its 
own expert commissions. As previously discussed by Steiner-Khamsi 
et al. (2020), mode 1 knowledge represents foundational research that is 
primarily concerned with advancing scientific discovery and disciplines. 
Mode 2 knowledge, meanwhile, refers to application-oriented, transdis- 
ciplinary, local, and involved expertise found outside purely academic 
settings (Gibbons et al., 1994; Nowotny et al., 2003). Without doubt, 
sector trends like open access to knowledge products, demand for 
evidence-based policy planning, and the pluralization of expertise (see 
Cairney et al., 2019; Eyal, 2019; Maasen & Weingart, 2005) have boosted 
the importance of mode 2 knowledge in this context. In Norway, this 
type of research is actively promoted: the so-called institute sector— 
including, for example, the Nordic Institute for Studies in Innovation, 
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Research and Education (NIFU, formerly NIFU STEP)—has become a 
significant knowledge producer in the education sector and is likely to 
expand in the near future. 

Closer analysis of the research publications listed in the White Papers 
here reveals the ministerial propensity to rely on mode 2 research. 
Comparing the bibliographies of the White Papers with those of the 
commission reports, the proportion of both national policy research and 
international policy research is noticeably higher in the White Papers 
(38.1096 vs. 22.8496 and 14.9796 vs. 4.896, respectively), while refer- 
ences to international academic research are roughly equivalent (around 
20% for each). In turn, the extent to which these references represent 
work published by members of the advisory commissions themselves 
needs to be investigated empirically. 

Second, the chairperson ofa given commission has a significant impact 
on its research orientation and knowledge production. In the Norwegian 
context, we considered the high-profile example of Sten Ludvigsen, who 
chaired the commission that produced NOU 2014:7 and NOU 2015:8. 
Ludvigsen is a professor in learning and technology at the University of 
Oslo. He played a pivotal role in the research-based evaluation of the 
2006 reform, organized by the Directorate for Education and Training 
and implemented between 2006 and 2012. Ludvigsen was also the leader 
of the program board of evaluation from 2008 to 2012. Through his 
leadership role in the evaluation, Ludvigsen gained a strong reputation as 
an expert and policy advisor within the sector before his appointment as 
commission chair. 

A Swedish analogy to Ludvigsen can be found in the figure of Jan-Eric 
Gustafsson, who chaired the commission that produced the Gathering for 
School Green Paper (SOU 2017:35),’ which led to the White Paper of the 
same name as part of the 2015/2018 reform. Gustafsson is a professor of 
education at the University of Gothenburg and is considered an expert in 
Swedish school reform, especially in the area of large-scale student assess- 
ments. Gustafsson is the most cited academic author, not only in the 
commission that he chaired, but also in the other Green Papers that the 
Ministry of Education and Research references in its White Paper. 

Figure 10.3 illustrates the author-reference network, where each author 
is connected to the documents that s/he authored and to their coauthors. 
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Fig. 10.3 Author-reference network. (Note: Circle: References. Square: Authors) 


A node size indicates how many references s/he served as an author. The 
map shows the top 15 authors who were most frequently cited. Among 
them, Gustafsson was the only individual researcher cited in the relevant 
SOUs and the Swedish White Paper. All other influential texts cited in 
the source documents were authored by institutions, including the 
Swedish National Agency for Education (Skolverket) and the Swedish 
Schools Inspectorate (Skolinspektionen), or by legal entities such as the 
Government (Regeringen) or Parliament (Riksdagen), as visualized in 
Fig. 10.3. 

In contrast to the frequent citation of Gustafsson’s work in the SOUs, 
Ludvigsen’s work was not cited once in the NOUs. This may seem sur- 
prising given his role as chair and his exemplary reputation as a policy 
advisor and scholar in educational studies (see Baek, 2020). Indeed, 
according to interviews with several members of the NOUs, Ludvigsen 
showed his commitment to evidence-based policymaking by distributing 
reading lists to commission members to ensure informed policy advice. 

The Ludvigsen commission appointed in 2013 consisted of eleven 
members.’ The commission was tasked to submit an interim report 
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including historical and comparative analyses of school subjects in pri- 
mary and secondary education, a feasibility study of international com- 
petence-based curriculum frameworks, and recommendations of national 
stakeholders in terms of students’ competency requirements. The interim 
report was delivered in 2014 (NOU 2014:7) and examined the knowl- 
edge on competencies from "various international organizations, educa- 
tion authorities in a number of countries and comprehensive research 
and report projects" (NOU 2015:8, p. 16). In addition, the commission 
appointed a research team that was tasked with providing an overview 
and making an assessment of different competency concepts and frame- 
works (see Erstad et al., 2014). The final report (NOU 2015:8) drew on 
both the interim report (2014:8) and the work of the research team and 
proposed a broad competency concept with four suggested areas as the 
foundation for the curriculum. 

In their role as chairs of their respective advisory commissions, 
Ludvigsen and Gustafsson publicly provided a scientific stamp of approval 
for OECD-informed national education policies. In Norway, however, 
the OECD link is a more complicated one: several international organi- 
zations and commercial actors had a bearing on the recommendations 
presented by the commission. These included the Partnership for 21st 
Century Skills, the International Society for Technology in Education, 
the EU and Key Competence Network on School Education, and Cisco, 
Intel, and Microsoft, who initiated the Meeting Assessment and Teaching 
of 21st Century Skills (see NOU 2014:7, Chapter 8). 

The two decision-making authorities of Norway and Sweden display 
marked differences in how they explain and justify the necessity for 
reform. In Norway, the Ministry of Education and Research draws on 
research (with a preference for mode 2 knowledge in the form of applied 
policy research), whereas the ministry in Sweden legitimizes its policy 
decisions with greater reference to national policy documents. This orien- 
tation toward research in Norway versus compliance in Sweden has been 
discernible at various levels of analysis. 
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The previous two sections examined the act of knowledge production 
carried out by the commissions and the ministries, respectively. In this 
section, we consider the application of this knowledge, guided by two key 
questions: To what extent do the respective Ministries use the knowledge 
provided by their expert commissions? And what kind of knowledge (or, 
more specifically, which references) from the NOUs and SOUS have been 
formally adopted in the White Papers? 

The answer to the first question is that significantly more knowledge 
from commission reports is used in Sweden than in Norway, but overall 
knowledge transfer is very scarce. As presented in Fig. 10.4, only 30% of 
the references used in the Norwegian White Papers are identical to those 
forwarded in the corresponding NOUS. The remaining 70% are novel 
references; the Ministry of Education and Research of Norway is very 
much an evidence-producer in its own right. We chose to describe this 
esoteric ministerial knowledge, entirely separate from the knowledge 
offered in the NOUS, as "political knowledge." 

In Sweden, the Ministry of Education and Research seems more 
inclined to adopt the knowledge sources used by the SOUS to support 
their assertions. Forty-four percent of the references in the White Papers 
are identical to those produced in the Green Papers. The two diagrams in 
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Fig. 10.4 Reference distribution in White Papers in Norway and Sweden 


10 How Much Is Policy Advice Changed and Lost in Political... 303 


Fig. 10.4 tell the ministerial, top-down side of the story; this perspective 
is explored further in the case study chapter on Sweden in Chap. 8 of this 
book. Yet we can equally invert the perspective and consider political 
translation from the bottom-up (i.e., from the viewpoint of the advisory 
commissions): Were the thorough reviews of policy documents, studies, 
and other relevant publications worth the time, effort, and resources? 
Was there any uptake at the higher political level? The question of whose 
knowledge (and which kind) is adopted at the political level is important 
since governments appoint numerous commissions whose output can 
then go unnoticed in terms of political uptake. 

Such uptake is barely visible in the Norwegian case: of the 2106 pub- 
lications amassed by the commissions to produce evidence for their 
reviews and recommendations, only 4% were then mentioned in the bib- 
liographies of the two White Papers. As illustrated in Fig. 10.5, this 
means that, on average, 96% of the expert knowledge references—gath- 
ered with great diligence and reviewed in great detail in the NOUs— were 
notexplicitly mentioned in the two White Papers ofthe 2016/2020 reform. 

The same pattern reemerges in the Swedish case, with only a slightly 
worse political uptake of commission report references. The bottom-up 
perspective suggests a very high cost of political transaction: only 38 doc- 
uments’ out of a possible 1373 cited in the SOUs (3%) reached the polit- 
ical level. Again, this suggests that a significant amount (97%) of expert 
knowledge has been lost in political translation. Given the highly selec- 
tive reception of commission reports at the political level (3% in Sweden; 
4% in Norway), the question becomes: which references did “make it”? 

The charts in Figs. 10.5 and 10.6 point to the fact that certain com- 
mission reports proved more attractive to their respective Ministry than 
others. In Norway, the Green Papers NOU 2014:7 (Pupils! Learning in 
the School of the Future: A Knowledge Base)" and NOU 2015:8 (The School 
of the Future: Renewal of Subjects and Competences)'' constitute the two 
reports with the greatest political influence. As noted above, both were 
the products of the Ludvigsen commission. More than any other, this 
commission was specifically tasked with identifying the areas where adap- 
tation of the existing curriculum was deemed necessary. 

Regardless of report length or number of references, NOU 2014:7 and 
NOU 2015:8 have clearly caught the attention of the Ministry of 
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Education and Research. In the 2020 reform, for example, NOU 2015:2 
includes a thorough literature review (703 publications) and is cited by 
both White Papers, yet the Ministry of Education and Research only 
considered 25 of them relevant at the decision-making level. 

Similarly, the Swedish Ministry of Education and Research appointed 
several commissions in preparation for the 2015/2018 reform. However, 
as shown in Fig. 10.6, not all of them had equal influence at the political 
level. As noted above, the Green Papers SOU 2017:35 and SOU 2016:94 
were especially integral to the Samling for skolan White Paper, not just 
to its title but also to the 2015/2018 reform as a whole. Together with 
SOU 2013:56, SOU 2017:35 is the only Swedish source document to be 
produced by a larger advisory commission; all the others came from one- 
person commissions, including SOU 2016:94 (Saknad/),'? chaired by the 
psychologist Malin Gren Landell. Such selective use of expert knowledge 
would appear to support the argument that appointed advisory commis- 
sions are becoming weaker (Lundberg, 2015), while the government at 
the same time grows in strength and independence in relation to its 
appointed advisory commissions (Petersson, 2016). 

We would be remiss if our descriptive bibliometric analysis excluded a 
qualitative review of the most politically influential commission reports 
in both countries. In Norway, the Green Papers NOU 2014:7 and NOU 
2015:8 have had the greatest impact in terms of political translation or 
knowledge transfer from the expert to the political level. The mandate of 
the Ludvigsen commission was to assess the degree to which the curricu- 
lum covers the competencies that students would need for the future, 
both professionally and socially. Unlike the Swedish case, the commission 
was not directly tasked with drawing on expert knowledge from the 
OECD. In fact, there was no need to do so, because the OECD 
competency-based Definition and Selection of Key Competencies cur- 
riculum framework (DeSeCo) had already been implemented in the 
reform of 2006 (see Steiner-Khamsi et al., 2020). This time, the question 
was whether, ten years later, adaptations and modifications needed to 
be made. 

The Ludvigsen commission supported the fundamental curriculum 
reform that was launched in 2006 and recommended that adaptations 
were made to reflect more recent educational frameworks and debates. It 
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considered its work an "advancement of the competence-oriented subject 
curricula today" (2015:8, p. 15). After reviewing several competency 
frameworks, notably, OECD's DeSeCo framework, Assessment and 
Teaching of 21st Century Skills (ATC21S), Partnership for 21st Century 
Skills (P21), Key Competences for Lifelong Learning (KeyCoNet), and 
National Educational Technology Standards (NETS), the Ludvigsen 
Commission recommended in its main report a broad concept of compe- 
tence comprising cognitive and practical skills as well as social and emo- 
tional learning and development (NOU 2015:8, p. 9). It proposed the 
following four areas of competence as the basis for setting priorities for 
school activities: subject-specific competence; competence in learning; 
competence in communicating, interacting, and participating; and com- 
petence in exploring and creating. These recommendations aligned with 
those of the international policy agenda and the OECD. Additionally, 
they built on learning sciences, and the commission emphasizes the 
importance of combining learning sciences and subject didactics research 
(Greeno, 2006, p. 46). Another key concept of the commission was cross- 
curricular competence; this particular recommendation, however, was 
not followed up in the White Paper. Instead, the Ministry explicitly stated 
that the renewal of the curriculum should focus on school subjects and 
not on cross-curricular competence (Ministry of Education and Research 
of Norway, 2016, p. 42). 

In Sweden, the two reports with the greatest political uptake of shared 
knowledge or references could be considered OECD reviews in national 
disguise. This applies to SOU 2017:35 in particular, where the commis- 
sion was explicitly instructed to make proposals based on the OECD 
Improving Schools in Sweden report given to the Swedish government in 
2015, a study itself triggered by Sweden's poor performance in the PISA 
tests. The OECD contextualization, therefore, lies at the core of the entire 
2015/2018 school reform process in Sweden. As for SOU 2016:94, the 
OECD influence is somewhat more indirect, given that the commission 
was not explicitly asked to draw on OECD expert knowledge. However, 
the OECD influence is present nonetheless: as early as the introduction, 
the report refers to student absenteeism as an important factor in declin- 
ing Swedish PISA performance. More specifically, the report points to a 
significant correlation between tardiness and students’ results in science 
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education (SOU 2016:94, p. 111). Hence, unlike the straightforward 
national adaptation of the 2015 OECD study in SOU 2017:35, SOU 
2016:94 implicitly draws on an OECD governance tool (the PISA test) 
to generate reform pressure. The measures to increase student attendance 
are instead couched in the strong national belief in equality, according to 
which the school system is charged with generating equal opportunities 
and life chances for all. Thus, rather than referring to OECD recommen- 
dations directly, in the case of SOU 2016:94, the Ministry of Education 
and Research leverages its policy-advisory system and cites a nationally 
adopted or "indigenized" version of an OECD recommendation. 


Summary and Conclusions 


In this bibliometric study, we compared policy documents relevant to the 
most recent school reforms in Norway and Sweden. The comparison 
focused on knowledge produced by the advisory commissions and the 
Ministries of Education and Research of the two countries. In particular, 
we were eager to understand the recourse to knowledge—in terms of 
frequency and type of knowledge used in references—in the reports of 
the advisory commissions (Green Papers) and in the ministerial decrees 
(White Papers). As well as knowledge production, we also investigated 
how much and which knowledge presented in the commission reports 
was actually taken up by the respective Ministry of Education and 
Research at the political level. In line with an earlier study (Steiner- 
Khamsi et al., 2020), we applied the concept of political translation to 
encapsulate the process of knowledge transfer from science to politics, 
from Green to White Papers, or from the advisory commissions to the 
Ministries of Education and Research. Naturally, this will always be a 
process in which some knowledge gets lost, rebalanced, and 
reinterpreted. 

In this study, we have considered at least three broader interpretations 
that help to theorize the policymaking process. Both deal to a certain 
degree with the observed repurposing of advisory commissions: (a) from 
a tripartite function (accountability, expertise, representation) to a singu- 
lar function (expertise); (b) as the first stage in a long process of 
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evidence-based policymaking; and (c) as domestic reviewers and transla- 
tors of global education policies. In this concluding section, we attempt 
to "de-Scandinavize" our findings and reflect on the larger phenomenon 
of network governance (Ball & Junemann, 2012) or polycentric gover- 
nance (Cairney et al., 2019). 

First, the traditional, tripartite purpose of advisory commissions— 
accountability, representation, and expertise—has dissipated, replaced 
with a solitary focus on expertise. Commissions are no longer appointed 
to keep the government accountable for its political decisions, but func- 
tion rather as prolonged arms of the Ministry of Education and Research. 
In Sweden, for example, the number of instructions that the advisory 
commissions receive from the Ministry of Education and Research has 
increased significantly over the past twenty-five years (Dahlstróm et al., 
2020). Some analysts see the close collaboration between commissions 
and the bureaucracy as an attempt to accelerate the pace of commission 
work and improve efficiency. Others regard the collaboration as too close 
for comfort, at risk of seriously undermining the independence of the 
commissions. 

Unsurprisingly, representation of diverse political perspectives has 
ceased to be one of the key requirements of commissions. As shown by 
Christensen and Holst (2017) in Norway and by Dahlstróm et al. (2020) 
in Sweden, political interest group representation in advisory commis- 
sions has decreased rapidly over the past twenty years. In Norway in par- 
ticular, academics and other researchers have filled the spaces left behind 
by the interest groups. Yet perhaps the most visible signpost of how advi- 
sory commissions have been repurposed is the advent of Special 
Investigator commissions in Sweden, also known as one-person 
commissions. 

The decline of social corporatism in Scandinavia is a well-documented 
and well-studied phenomenon in political science. The traces are equally 
discernible in the advisory commissions in the education sector: our 
research has shown that we need only focus on the composition of their 
members. Compared to earlier compositions that reflected a greater, 
broader representation from various interested parties, the advisory com- 
missions have indeed become depoliticized and are now narrowly charged 
with producing bodies of “evidence.” 
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Second, the expertization of advisory commissions does not imply a 
depoliticization of the policymaking process as a whole. On the contrary, 
as mentioned in the introductory chapter, interest groups now exert their 
political influence elsewhere, most notably during the stakeholder review 
process (stage 2) and in standing committees or parliamentary commit- 
tees (stage 4), as shown in Fig. 10.1. In an effort to refine the definition 
of the expertization of advisory commissions, we suggest the adoption of 
a multi-level perspective that brings the entire policymaking process into 
focus. In particular, it may be useful here to rephrase the five stages of the 
policymaking process in terms of evidence-based policymaking. The 
same figure presented in the introductory section (see Fig. 10.1) may be 
depicted in terms of the binary between science and politics, with the 
collection of scientific evidence assigned to the advisory commission (left 
side) and the issuing of a decree or legally binding act by the legislative 
body (right side). Figure 10.7 depicts this binary in greater detail. 

Nevertheless, the sequence of evidence-based policy planning depicted 
above—starting with the gathering of evidence in commission and end- 
ing with evidence-based bills or parliamentary acts—is not as linear in 
actual practice as the arrow would have us believe. In reality, until a new 
bill or act is actually passed, a multitude of policy actors are involved. 
These include experts in advisory commissions; stakeholders, interest 
groups, and civil society during the referral process; the executive body of 
government, that is, the Ministry of Education and Research; political 
parties and interest groups that exert influence on the parliamentary/ 
standing committees; and the legislative body in the form of parliament. 
Thus, the same reform idea (competency-based curriculum reform in 
Norway, strengthening student performance in Sweden) “morphs as it 
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Fig. 10.7 The premise of evidence-based policymaking 
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moves” (Cowen, 2009) at each subsequent stage of the policymaking 
process. With each transfer from one level to the next, both science and 
politics come into play, and there is no smooth continuum from one to 
the other. Rather, the left-to-right sequence is “interrupted” by several 
rounds of additional information-gathering (scientific) and consultation 
(political). As a result, evidence-based policymaking occurs in an iterative 
and ubiquitous manner that cannot be neatly confined to a designated 
stage in the policymaking process. 

Two examples may suffice here. On the one hand, the production of 
evidence, a role designated to the advisory commissions, also occurs in 
White Papers. In fact, as shown in the political translation section of this 
chapter, the two Ministries incorporate remarkably few references from 
their advisory commissions and instead draw on their own bases of 
knowledge. A meager 3% and 4% of the knowledge sources or references 
amassed by the advisory commissions in Sweden and Norway, respec- 
tively, are actually used by the Ministry of Education. As illustrated in 
Table 10.2, most of the knowledge referenced in the White Papers is 
home-spun, taken from domestic policy documents. This knowledge 
qualifies as “regulatory science” (see Eyal, 2019) or mode 2 knowledge; in 
other words, advisory commissions are not the only entity in the policy- 
making process that produces knowledge. Different kinds of knowledge 
and “evidence” are produced by different actors at the various stages of 
the policymaking process, and this “evidence” is produced, translated, 
and changed at each subsequent level of the policymaking process. 

On the other hand, political coalition- and consensus-building is not 
restricted to the last two stages (parliamentary/standing committees and 
parliament) of the policymaking process, but also occurs at the earlier 
stages as well. In particular, the referral process or the stakeholder review 
(known as the “hearing” in Norwegian and Swedish contexts) is meant to 
ensure democratic participation in the policymaking process. For exam- 
ple, the hearing for the school subject Norwegian Language—conducted 
in March- June 2019 as part of the renewal of the Knowledge Promotion 
Reform (2016/2020)—yielded a total of 1074 replies (UDIR, 2019). 
Similarly, political viewpoints determine to a certain extent which contri- 
butions from which members of a standing committee are considered as 
valid "evidence." In sum, different kinds of "evidence" are produced at 
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each and every stage of the policymaking process, and actions taken are 
political, sometimes more overtly than at other stages. It is therefore 
important to juxtapose the premise of evidence-based policymaking 
(Fig. 10.7) with its actual practice. 

As shown in Fig. 10.8, information-gathering and consensus-building 
occurs in practice at each and every step of the policymaking process, 
blurring the line between science and politics. 

A good case in point are advisory commissions that, despite their man- 
date to find facts and gather information, build consensus with stake- 
holders outside the commission as well as among its constituent members. 
In Norway, the informal collaboration with the teachers union illustrates 
this consensus-building process with external stakeholders: even though 
the teacher unions were not formally represented in the Ludvigsen com- 
missions, the Union of Education Norway explicitly expressed its sup- 
port for the commissions work (Utdanningsforbundet, 2015). Likewise, 
the commission made a point of highlighting its regular meetings with 
the teacher unions as well as with other interest groups and stakeholders 
(NOU 2015:8, p. 17). In an effort to reach out to stakeholders and the 
general public, the commission even set up its own blog. Several experts 
have interpreted this novel outreach approach as a sign of network gover- 
nance, in which the perspectives of various stakeholders are informally 
incorporated at an early stage to secure broad political support later in the 
process, when the report and the recommendations are released. These 
informal collaboration networks also demonstrate the crucial role of the 
chairs of the commissions, who may decide whether extra effort is made 
to include additional experts and stakeholders not otherwise represented 
in the commission. 


Science 


Politics 


Commissions Edutetiefi and a 
Research Committees 


Fig. 10.8 The practice of evidence-based policymaking 
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In addition to securing political support from diverse stakeholders 
outside the commission, a Green Paper is in and of itself a compromise 
and an outcome of negotiations among the members of the advisory 
commission. In his empirical study of expertise-seeking arrangements in 
policymaking, Baek (2020) interviewed twelve policy experts who had 
served on Norwegian expert commissions. 'Ihe participants shared that 
viewpoints that were considered too radical to be accepted at the ministe- 
rial and parliamentary levels were sometimes left out from the “con- 
sented" reports. Furthermore, the consensus-building process was seen to 
be influenced by the existing hierarchy among the committee members. 

A closer examination of evidence-based policymaking demonstrates 
that, rather than conceiving of science and politics as binary opposites, it 
is more accurate to consider them as structurally coupled. The overlap 
between science and politics not only makes the policymaking process 
non-linear and messier, but also creates room for non-state actors to par- 
ticipate in the policymaking process, including experts, users, profession- 
als, and (in Sweden more so than in Norway) businesses. 

Third, we have shown that a transnational lens is indispensable to 
understanding why and how advisory commissions have become repur- 
posed in an era of global education policies. Our findings clearly indicate 
that the most influential Green Papers in both countries were those that 
reviewed OECD recommendations and, in the case of Norway, com- 
pared OECD recommendations (the DeSeCo framework) with other 
international competence-based curriculum frameworks. Thus, the advi- 
sory commissions acted as important bridges between global and national 
reform debates, helping to translate and adapt global education policies, 
or more narrowly OECD policies, into a national setting. Drawing on 
the multitude ofavailable international curriculum frameworks, or global 
education policies more broadly, as sources of authority with which to 
back up national agendas is not out of the ordinary. What is striking, 
however, is the preference of national policymakers for a very particular 
kind of international knowledge: one that provides metrics, relies on 
international comparisons, and is published by a cluster of affluent coun- 
tries. In Europe, government officials are especially receptive to OECD 
data, studies, and recommendations (Grek, 2017; Niemann & Martens, 
2018; Ydesen, 2019). In this study, we have seen how OECD 
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recommendations were elevated to a gold standard explicitly in Sweden 
as part of the 2015/2018 reform and implicidy in Norway as part of the 
2006 curriculum reform. 

We conclude this study with three observations. First, the advisory 
commissions in Norway and Sweden have been repurposed in ways that 
place greater emphasis on expertise at the expense of accountability and 
representation. Second, multi-level analysis reinforces the notion that 
these commissions represent only one stage in a long sequence of 
evidence-based policymaking. Finally, the transnational perspective helps 
us to see how advisory commissions have more recently been used as 
bridges between global and national policy arenas. In particular, they can 
review, translate, and recontextualize OECD recommendations—or, in 
the case of the recent reform in Norway, OECD and other international 
frameworks—to fit into the specific national context. 


Notes 


1. In Norwegian and Swedish, a White Paper is called Melding til Stortinget 
and Proposition, respectively. 

2. Translation of the full title: Subjects, in-depth learning, understanding. 
Renewal of the Knowledge Promotion Reform. In Norwegian: Fag— 
Fordypning—Forstaelse—en fornyelse av Kunnskapsloftet (Report No. 28 
to the Norwegian Parliament, 2015/16). 

3. Translation of the full title: Eager to learn: Early intervention and quality 
in schools. In Norwegian: Lerelyst—tidlig innsats og kvalitet i skolen 
(Report No. 21 to the Norwegian Parliament, 2016/17). 

4, In Swedish: Tydliga mal och kunskapskrav i grundskolan. 

5. SOU 2017:35. Gathering for school: A national strategy for quality and 
equivalence. Final report of the 2015 School Commission. In Swedish: 
Samling for skolan—Nationell strategi for kunskap och likvärdighet. 
Slutbetänkande av 2015 års skolkommission. 

6. SOU 2016:66 contained no references. 

7. Samling for skolan—Nationell strategi for kunskap och likvärdighet. 
Slutbetünkande av 2015 ars skolkommission. 

8. Sten Ludvigsen, Professor/Head of the Committee; Eli Gundersen, 
Chief Municipal Education Officer; Sigve Indregard, journalist; Bushra 


10 How Much Is Policy Advice Changed and Lost in Political... 315 


Ishaq, social commentator; Kjersti Kleven, Chairperson of the Board of 
the Federation of Norwegian Industries; Tormod Korpas, Head of an 
upper secondary school; Jens Rasmussen, Professon Copenhagen, 
Denmarl; Mari Rege, Professor; Sr Sunniva Rose, PhD candidate; 
Daniel Sundberg, Professor, Vàxjó, Sweden; Helge Oye, project manager. 
9. The total of shared references in Fig. 10.5 exceeds thirty-eight due to 

identical texts cited in multiple commission reports. 

10. NOU 20147. Elevenes læring i fremtidens skole: et bunnskapserunnlag. 

11. NOU 2015:8. Fremtidens skole: fornyelse av fag og kompetanser. 

12. WP (2017/18). Gathering for school: A national strategy for quality and 
equivalence. Final report from the 2015 School Commission. 

13. SOU 2016:94 Missing! Pay attention to student absence and take action. 
Report on turning absenteeism into attendance, an investigation into 
problematic student absenteeism. 
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The OECD and the Field of Knowledge 
Brokers in Danish, Finnish, and Icelandic 
Education Policy 


Christian Ydesen, Jaakko Kauko, 
and Berglind Ros Magnusdottir 


Reflecting on his long-time service in the Organization for Economic 
Cooperation and Development (OECD), the Norwegian social econo- 
mist Kjell Eide (1925—201 1)! described the historical relations between 
the Nordic countries and the OECD in education. Writing about the 
1960s, Eide (1990) contended that “the Nordic countries were in a period 
of strong expansion and reform, and there too, it was at times valuable to 
have the OECD’s blessing for the political directions underlying the 
reforms” (p. 20). One of Eide’s main points in his 1990 essay was that the 
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Nordic countries and the OECD have had a close relationship in educa- 
tion for a long time. He even hinted that the OECD has played the role 
of a knowledge broker in the Nordic region. 

Recent research has painted the same picture. In their comparative 
analysis of education policies in the Nordic region, Dovemark et al. 
(2018) emphasized how OECD country reviews and other expert 
reports—often commissioned by the national governments—are used to 
“legitimize economic and strict educational policy decisions" (p. 125). In 
her recent analysis of the OECDS role in the governing of education in 
Sweden, Grek (2020) argued that the OECD takes up a position as a 
boundary organization "constructing a very carefully maintained equilib- 
rium amongst the different powers and interests of the actors in the field" 
based on “a hybrid of both knowledge and policy closely intertwined.” 
More specifically, the hybrid consists of “hard numbers, administrative 
advice, managerial know-how and best practice recommendations in a 
big, versatile, complex and ever-changing mixture of facts and values" 
(p. 17). Important nuances to these pictures emerge from the previous 
chapters in this volume. 

In this chapter, we follow this trail of research into the OECD—Nordic 
region relations in education policy by applying a specific focus on the 
relations between the OECD and national knowledge brokers. In other 
words, the chapter investigates the extent to which the OECD via its 
relations with national institutions has infused policy change in the 
Nordic region. As such, we do not focus on the OECD as an actor that 
impacts national school reform; rather, we examine the relation between 
the OECD and national policy actors that, at critical stages, draw on the 
authority of the OECD to develop and substantiate their own national 
reform strategies. The chapter offers an in-depth analysis of Denmark, 
Finland, and Iceland as empirical cases to understand the nexus or assem- 
blage of the OECD with national institutions serving as knowledge bro- 
kers in the Nordic region. 

The relevance of this perspective is supported by the fact that the 
OECD does not have the mandate or the ability to dictate policies in 
member countries. As many researchers have noted, the OECD operates 
with a distinct soft power mode of governance (Bieber & Martens, 2011; 
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Mundy et al., 2016; Steiner-Khamsi, 2019). One example of this soft 
governance is the peer pressure associated with multilateral surveillance 
among member countries in the OECD. In this respect, Morgan and 
Volante (2016) pointed out that "the OECD has pursued a strategy of 
‘soft’ persuasion that naturalizes the idea that performance in a series of 
measurement exercises represents educational quality" (p. 778). This 
strategy is underpinned by what John Krejsler (2019) has called a "fear of 
falling behind" among PISA-participating nations. 

In terms of education policy reforms, the previous chapters in this 
book have amply demonstrated that a number of national institutions 
serve as key providers of knowledge. They serve as arbiters, brokers, pro- 
ducers, and mediators of knowledge and policy flows between transna- 
tional, national, and local spaces. From an OECD perspective, they 
might even be described as bridgeheads or intermediaries for the dissemi- 
nation and impact of OECD policy recommendations and policy 
instruments. 

In a theoretical sense, this observation might be expressed using the 
concept of “instrument constituencies” (Béland & Howlett, 2016). 
According to Simons and Vof (2018), 


Policy instruments [...] are not only “active” or “alive” because they con- 
tain scripts for reordering society [...] but also because they gather a con- 
stituency comprised of practices and actors oriented towards developing, 
maintaining and expanding a specific instrumental model of govern- 
ing. (p. 31) 


In their analysis of the OECD, Verger et al. (2019) drew on the same 
concept using Kingdon’s (2003) terms. Verger et al. (2019) pointed out 
how the potential for the OECD governance mechanisms to advance 
agendas is rooted in a “[...] capacity to open a policy window through 
which the problem, policy, and politics streams are affected in a relatively 
coordinated and coherent way” (p. 236). In this sense, the chapter con- 
tributes to our understanding of the OECD as a policy actor—even 
though that is not our specific focus—because we unpack the role and 
relations between the OECD and national institutions in the three case 
countries. 
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Arguing the Relevance and Context 
of the Three Case Countries 


The following sections outline a brief frame of interactions between the 
OECD and the three case countries. This outline serves as the argument 
for selecting the three case countries. 

The OECD has a history of influencing and making recommendations 
for the Danish field of education (Ydesen, 2021). For instance, in April 
1963, the Danish Ministry of Education established an economic and 
statistical section in response to an OECD request in the program for 
Educational Investment and Planning (Ydesen & Grek, 2019). In 2004, 
an OECD report, produced at the request of the Danish government, 
found that Danish education research was too unfocused and called for 
the establishment of a clearinghouse for educational research in Denmark 
(Krejsler, 2017; OECD/CERI, 2004). The Danish Clearinghouse for 
Educational Research was established in 2006. The report also empha- 
sized the importance of establishing an evaluation culture, which led to 
the implementation of national testing in compulsory education in sub- 
sequent years (Shewbridge et al., 2011). Another key initiative following 
from the 2004 OECD report was the formation of the School Agency, 
which had an explicit focus on evaluation culture and improving quality 
in the public school system. In 2012, the OECD identified Denmark as 
one of only three countries where the PISA results have had an “extremely” 
big impact on educational policies and practices. 

The OECD has been a frequent collaborator in Finnish education 
policy as well, and Finland has received additional international attention 
due to its high-scoring performance in PISA. As described in Chap. 5, 
researchers investigating Finnish education disagree on how direct the 
influence of the OECD on national education policy is. It is clear that the 
main higher education reforms in Finland have been preceded by OECD 
reviews (Kallo, 2009) and that there is an element of using the OECD as 
a clearinghouse for higher education reforms (Kauko, 2011); however, 
the effect of the OECD on primary and secondary education is more 
debatable. In understanding this effect, PISA has been the focus of 
research. Sahlberg (2011) argued that PISA success has resulted in a lack 
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of innovation in primary and secondary education. However, Seppanen 
et al. (2019) found that Finnish governments have been noticeably active 
in comprehensive school policies during the new millennium. Kauko 
et al. (2021) argued that PISA has been compartmentalized from national 
reviews and thus has limited effect. Rautalin (2013) pointed out how 
results were used to strengthen the interest groups and government off- 
cials’ views with little media criticism. 

The OECD has a long history of influencing education in Iceland, 
starting explicitly in the 1960s, when the minister of education intro- 
duced a human capital approach through an extensive examination of the 
education system (Guttormsson, 2008, pp. 88-89). This examination 
was one of the building blocks of the comprehensive schooling act in 
1974. The OECD began undertaking examinations of compulsory 
schooling in 1986 (Guttormsson, 2008, p. 264f). The OECD’s most 
apparent influence on Icelandic education is through the PISA measure- 
ments that have substantially influenced Icelandic educational discourse 
for the last 18 years. After the financial crash in 2008, the economic and 
governmental system was highly criticized (Oddsdóttir, 2014). In recent 
years, Iceland has strived to rebuild its education system. Ihe OECD has 
played a role in shaping the discourse in recent educational policy papers 
that have strived for a more professional and transparent system. 


Research Questions 


Building upon these exemplary connections and policy flows, we hypoth- 
esize that there are very strong interactions between Denmark, Finland, 
and Iceland as cases and the OECD in education. Building on the find- 
ings of the national chapters in this volume, the objective of the chapter 
is to analyze policy flows between the OECD and the three case countries 
and analyze the political capital created by the OECD and its use in the 
national contexts. More specifically, we aim to investigate the gearing, 
entry points, and interactions in the links between the OECD and 
national institutions in infusing policy change. 

In terms of policy reforms, we focus on the same reforms as have been 
analyzed in the respective national chapters in this volume. In this sense, 
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the chapter offers supplementary insights into the other chapters of this 
volume through analyses of the transnational policy flows and the knowl- 
edge brokers behind the education reforms in each case country. In pur- 
suing this aim, we follow the guidance of three research questions: 


1. Which policy instruments connect the OECD with each national 
context? 

2. Which national institutions are the central providers of evidence for 
national education reforms? 

3. How are these institutions located in the national fields of education, 


and to what extent do they serve as knowledge brokers between the 
OECD and the national contexts? 


Methodology and Chapter Structure 


Our methodological approach takes a starting point in the Foucauldian 
idea about bringing knowledge and power into one analytical field, 
assuming that these two are connected and in interrelation molding each 
other (Popkewitz & Brennan, 1998). We treat institutions and experts as 
agents who are positioned in a privileged way that allows them to be the 
providers of seemingly objective knowledge underpinning education 
reforms while at the same time exerting and institutionalizing power rela- 
tions and power discourses in the political field of education reforms. 
Understanding the workings of this mechanism is vital for understanding 
the nexus between the OECD and the Nordic region. 

The methodological recipe employed in the chapter consists of three 
analytical steps aligned with the three research questions. The first step 
pinpoints the central policy instruments connecting the OECD with 
each national context and identifies the central institutions associated 
with these policy instruments. According to Lascoumes and Le Galés 
(2007), a policy instrument may be defined as: 


a device that is both technical and social, that organizes specific social rela- 
tions between the state and those it is addressed to, according to the repre- 
sentations and meanings it carries. It is a particular type of institution, a 
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technical device with the generic purpose of carrying a concrete concept of 
the politics/society relationship and sustained by a concept of regula- 
tion. (p. 5) 


In the case of the OECD, the main policy instruments are policy reviews 
(e.g., country or thematic reviews), global progress reports (e.g., Education 
at a Glance), and international large-scale assessments (e.g., PISA and 
TALIS). Following the definition above, such policy instruments reso- 
nate in the national institutions, allowing knowledge brokers to operate 
between the OECD and the Danish, Finnish, and Icelandic contexts 
(e.g. universities, sector research institutions, and consortia). 

The second step relies on descriptive statistics of bibliometric analysis 
and content analysis of national and Nordic policy documents. This step 
serves the purpose of identifying the significance and centrality of the 
national institutions associated with the OECD. We use documents from 
three different reforms in Denmark, Finland, and Iceland. The biblio- 
metric data is from white and green papers in each context. We draw on 
the policy instruments identified in the first step and analyze how the 
national institutions are engaged in the translation of these instruments 
into the national contexts. 

The third step employs a contextual analysis of the institutions and 
agents to locate them in the respective national field of education. We use 
open sources to conduct the analysis of key institutions and agents in the 
respective national fields (Menashy & Verger, 2019). Finally, we look at 
the significance and position of these national institutions and the OECD 
in the national educational field. 

In the concluding discussion, we comparatively look across the three 
cases and offer insights into the research questions. In this sense, the 
chapter illuminates the configuration and workings of the OECD- 
centered epistemic community forming the modes of knowledge and 
governance woven into the Nordic education fabric. 
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Denmark: A Contested Field of Education 


Evidence and Research 


Looking at the most central OECD policy instruments playing into the 
Danish reform process of 2013, it is clear that global progress reports and 
international large-scale assessments (ILSAs) are most prevalent in terms 
of citations. Ihe OECD was the most frequently cited international pub- 
lisher, with the most important document being the 2009 PISA results 
(OECD, 20102). Thus, OECD policy instruments served as important 
points of orientation among key agents in the Danish education pol- 


icy field. 


Despite the large number of studies in Table 11.1, the OECD was 
only the fifth most cited publisher in the Danish policy documents. 


Table 11.1 Cited documents published by the OECD in the Danish database 


Document title in categories 


Policy reviews 

Strong Performers and Successful Reformers in 
Education—Lessons from PISA for the United States 

Preparing Teachers and Developing School Leaders 
for the 21st Century 

OECD Reviews of Evaluation and Assessment in 
Education—Denmark 

Global progress reports 

Education at a Glance 2010 

International large-scale assessments 

PISA 2009 Results: What Students Know and Can Do: 
Student Performance in Reading, Mathematics and 
Science (Vol. I) 

PISA 2009 Results: Overcoming Social Background. 
Equity in Learning Opportunities and Outcomes 
(Vol. II) 

PISA 2009 Results: What Makes a School Successful? 
Resources, Policies and Practices (Vol. IV) 

OECD Review on Evaluation and Assessment 
Framework for Improving School Outcomes. Design 
and Implementation Plan for the Review 

Creating Effective Teaching and Learning 
Environments—First Results From TALIS 

PISA 2009 Results: Students Online (Vol. VI) 


Year of 
publication 


2011 
2012 


2011 


2010 


2010 


2010 


2010 


2009 


2009 


2011 


Number of 
citations 
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Clearly, a number of other knowledge providers were at play in the 
2013 Danish education reform. Three of the knowledge providers that 
were cited more often than the OECD were government organizations, 
namely the Ministry of Education itself and government-funded sector 
research institutions (see Chap. 4 in this volume). They were responsi- 
ble for no less than two-thirds of all citations in the policy documents 
(Fig. 11.1). 

The role and significance of the OECD as a knowledge provider can- 
not be determined by the number of citations alone. The OECD also 
influenced the mindset and the constituency surrounding the reform 
process. A key point of orientation here appears in the knowledge brokers 
between the OECD and the Danish education field, which can be found 
in the shifting consortia tasked with conducting the PISA surveys. At the 
time of the reform, the Danish PISA consortium consisted of the Danish 
School of Education (Aarhus University), Statistics Denmark, and the 
Danish Institute of Government Research (AKF). The consortium 
included a university, a sector research institution, and the national statis- 
tical service, all based in the greater Copenhagen area. 


Most cited publishers 


60 O- s57 o 
500 ° 
400 ° 
300 ° 
200 o 
16 
u | [ [ E l 
The Ministry EVA OECD SAGE Aarhus Oxford Dafolo 
of Education Publications University arian University 


Press 


Fig. 11.1 Most cited publishers in the Danish policy documents 
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The chairman of the PISA consortium since 2000 was Professor Niels 
Egelund from the Danish School of Education. Egelund was the leading 
figure behind the report Danske unge i en international sammenligning 
[Danish Youth in International Comparison], which reported on the 
results of PISA 2009, and he was a member of the School Agency chair- 
manship. Another leading figure was Professor Lars Qvortrup, who at the 
time was dean of the Danish School of Education and who worked 
closely with Egelund. It is striking that Egelund, Qvortrup, and their col- 
leagues, Professor Jens Rasmussen and Andreas Rasch-Christensen, head 
of research at VIA University College, served on a number of ministerial 
committees and institutions surrounding the reform. Generally, these 
four prominent agents, who command considerable capital in the Danish 
field of education, have been very vocal and visible in the whole reform 
process, beginning with the preparatory work and continuing to the eval- 
uation of the reform. 

However, looking at the configurations of the field of education 
research in Denmark paints a picture of a rather acrimonious research 
environment. The 2013 school reform has been a particular bone of con- 
tention. The researchers mentioned above, who were associated with 
OECD policy instruments and the development of education policy, 
constitute one camp in the field, whereas a host of critical researchers 
make up another camp. Most notably, in his PhD dissertation, Keld 
Skovmand (2017) claimed that the 2013 education reform was not 
grounded in evidence or knowledge. These debates are still ongoing some 
seven years after the reform was implemented. Thus, it is fair to say that 
Danish education research often finds itself in a very toxic environment 
with significant antagonism between at least two main clusters, one being 
the evidence-based what-works type of research and the other being 
research adhering to pedagogical ideals about Bildung and emancipation 
as well as a notion of pedagogy being a unique field with its own values 
and contributions (Romer, 2017). 

In this environment, the Ministry of Education has followed its own 
agendas and priorities without engaging or siding explicitly with one 
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camp or the other. Being preoccupied with these agendas and priorities, 
the Ministry has fallen short of making explicit connections between 
OECD policy instruments and Danish education reform. This somewhat 
retired role has provided ample space for professional and academic 
debates to unfold—and perhaps for the trenches to be dug deeper. 

A combination of these insights with the findings in Chap. 4 of this 
volume indicates that the OECD lent authority to knowledge by provid- 
ing political capital through evidence. This capital was picked up—or 
extended—to power national agents who were able to shape the Danish 
education agenda in accordance with the ambitions and instruments fea- 
tured in the 2013 reform. 


Finland: State-Centered Production of Data 


The OECD policy instruments under scrutiny in this chapter most rele- 
vant for Finland's education system are OECD policy reviews and inter- 
national large-scale assessments. Research has documented more national 
policy changes in relation to the former, while researchers have seen the 
latter as serving more a legitimation purpose (e.g., Rautalin, 2013; Rinne 
et al., 2004; Sahlberg, 2011; Seppänen et al., 2019). In the OECD elec- 
tronic archive? which starts from 2005, there are three policy reviews 
that discuss primary and secondary education in Finland: a thematic 
review on equity in education (OECD, 2005), a country case study on 
digital learning resources (OECD, 20082) as part of a Nordic report 
(OECD, 2009), and in a school leadership report where Finland was one 
case country (Pont et al., 2008). There is also one influential (Kallo, 
2009; Kauko & Diogo, 2011) review from 2006 (OECD, 2006) address- 
ing tertiary education. Finland features in the global progress reports 
entitled Education at a Glance, and the country has participated in all 
OECD ILSAs apart from TALIS in 2008 (Sivesind, 2019). Finland was 
first in PISA in reading (2001), mathematics (2003), and science (2006) 
before dipping slightly in reading to second (2009), in science to fifth 
(2015), and more dramatically in mathematics to 12th (2012; OECD, 
2002, 2004, 2006, 2010b, 2014, 2016). While the results might have 
caused debates resulting in dramatic changes in other countries, the 
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provision of education and the basic principles of the comprehensive 
school have remained the same in Finland. One of the biggest reforms 
has been the Core Curriculum Reform of 2014, as described in Chap. 5. 
In this subsection we analyze the curriculum reform, drawing on the bib- 
liometric database before we scrutinize the OECD reports and analyze 
the networks working with ILSAs. 

The findings from our bibliometric analysis reveal that the most impor- 
tant sources of knowledge for the curriculum reform in 2014 can be 
divided into five main groups. Table 11.2 displays all publishers with 
more than ten references, accounting for 51% of all Finnish references in 
the database. The most important group contains government organiza- 
tions, specifically the Finnish National Agency for Education and the 
Ministry of Education and Culture. These organizations have published 
around one-third of all references in the Finnish documents. The next 
group includes two universities, the University of Jyvaskyla and the 
University of Helsinki, with around 1296 of all references. The third 
group comprises national and international publishers, focusing on both 
popular and science publications (PS Publishing, Werner Sóderstróm 
Limited Company WSOY) or only on science (Taylor & Francis). This 
group is responsible for publishing 696 of the referenced material (35 


Table 11.2 Most cited publishers in the 2014 Finnish curriculum reform green and 
white papers 


% of all 
Publisher [Finnish Name] Count references 
Finnish National Agency for Education 170 23 
[Opetushallitus] 
Ministry of Education [and Culture] [Opetus- ja 57 8 
kulttuuriministerió] 
University of Jyvaskyla [Jyváskylàn yliopisto] 55 8 
University of Helsinki [Helsingin yliopisto] 32 4 
Taylor & Francis 16 2 
PS Publishing [PS-kustannus] 15 2 
OECD Publishing 14 2 
Werner Sóderstróm Limited Company [WSOY] 11 2 
Sum 370 51 
Total Finnish References in Database 729 100 


Note: If a document has been cited many times, all citations are counted 
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references). The fourth group consists of OECD Publishing, with 14 ref- 
erences (2%). In summary, over 60% of all references in the Finnish data- 
base were published either by state or university actors, and the focus of 
this chapter, the OECD, played a minor role in the number of direct 
references. 

Table 11.3 lists the 14 OECD-published documents that are refer- 
enced in the 2014 Finnish curriculum reform. Half are directly linked to 
numeric education indicators (i.e., global progress reports and ILSAs); 


Table 11.3 Cited documents published by the OECD in the Finnish database 


Year of Number of 
Document title in categories publication citations 
Policy reviews 
21st Century Skills and Competences for New 2009 1 
Millennium Learners in OECD Countries 
Career Guidance and Public Policy: Bridging the Gap 2004 1 
The Definition and Selection of Key Competencies: — 2005 1 
Executive Summary 
Transition from Initial Education to Working Life. 2000 1 
Making Transitions—Work, Education and Skills 
Trends Shaping Education—2008 Edition 2008 1 
Understanding the Brain: The Birth of a Learning 2007 1 
Science 
Why Career Information, Guidance and Counselling. 2002 1 
Matter for Public Policy Working Draft no 1. 
7.1.2002 
Global progress reports 
Education at a Glance 2007 2007 1 
Education at a Glance 2009: OECD Indicators 2009 1 
International large-scale assessments 
Learning for Tomorrow's World. First Results from 2004 1 
PISA 2003 
Luonnontieteiden, lukemisen ja matematiikan 2006 1 
osaamisen arviointi PISA 2006 viitekehys. 
[Assessing Scientific, Reading and Mathematical 
Literacy: A Framework for PISA 2006] 
PISA 2009 Results: Executive Summary 2010 1 
PISA 2009 Results: What Students Know and Can 2010 1 


Do—Student Performance in Reading, 
Mathematics and Science 

Programme for International Student Assessment 2009 1 
(PISA) 
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specifically, five are PISA-related, and two are linked to Education at a 
Glance. Apart from one source (Understanding the Brain), all policy 
reviews were closely linked to the content of the curriculum reform: 
skills, competences, future projections, and career guidance. 

Outside the reference database, when looking at the background of all 
reports on Finland published by the OECD, the social networks reveal a 
more nuanced picture. The background information of OECD policy 
reviews makes it clear that the main informants for the OECD teams 
come from ministries, universities, and interest groups. In the visiting 
program of the two review teams (OECD, 2005, 2008a) and the work- 
ing group writing the background memo (Ministry of Education, 2007) 
for the leadership report (Pont et al., 2008), 86 names? of informants are 
mentioned. The main groups are 24 officials from ministries or the 
National Agency for Education, 16 university researchers (mainly from 
the University of Helsinki or University of Jyvaskyla), 12 labor market 
organization representatives, 10 education interest group representatives, 
8 schoolteachers or principals, and 6 representatives from cities or munic- 
ipalities. More women (z = 49) than men (z = 37) were interviewed by 
the review teams. The number of people interviewed does not necessarily 
communicate the impact of single institutions on the report, as many of 
the interviews happened in groups. 

When considering OECD ILSAs, two institutions and communities 
of experts are important. The Ministry of Education and Culture con- 
tracted the implementation of PISA to either one or both of two organi- 
zations: the Finnish Institute of Educational Research at the University of 
Jyvaskyla and the Centre for Educational Assessment at the University of 
Helsinki. The former has also been responsible for TALIS. During and 
after the curriculum reform, the contractor has been a consortium of 
these two (Opetus- ja kulttuuriministerió, 2013, 2015, 2018). The uni- 
versities of Jyvaskyla and Helsinki and their university research centers 
seem to be main hubs for OECD data expertise in Finland. They were 
most relevant in the OECD review visits, and they are also responsible 
for implementing ILSAs in Finland. The connection seems to be institu- 
tional: the names of the experts interviewed for the reviews and those 
conducting ILSA research do not overlap except for Professor Jouni 
Valijarvi, who was the head of the Finnish Institute of Educational 
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Research until 2017. Nevertheless, these two universities are also the 
main publishers featured in the evidence production for the curriculum 
reform analyzed in Chap. 5. 

When these arrangements are considered together, the picture of the 
main institutions for providing knowledge for decision-making in educa- 
tion starts to unfold. 'The production of knowledge in the case of the 
curriculum reform was much aligned with the picture of the country 
review visits. As noted in Chap. 5 and as we have seen in the Danish case, 
there is a distinct state-centeredness in data production through the 
involvement of the Finnish National Agency for Education and the 
Ministry of Education and Culture. In addition to these, the University 
of Helsinki and the University of Jyvaskyla stand out from the aca- 
demic side. 


Iceland: OECD as a Leading External Source 


Iceland features a well-established and systematic state-centered produc- 
tion of OECD data for all school levels. The Ministry of Education, 
Science, and Culture is a key institution that has issued many study 
reports that played an essential role in the policy papers from 2013 to 
2017 along with the OECD ILSA documents. The previous National 
Centre for Educational Evaluation was the main knowledge broker 
between the OECD and Iceland concerning ILSA documentation on 
PISA from 2000 and TALIS from 2008. In 2015, the institutional struc- 
ture in the field of education was reformed when the National Centre for 
Educational Evaluation and the National Centre for Educational 
Materials merged into one institution, the Directorate of Education (act 
of law nr. 91, 2015). Currently, the production of OECD ILSA docu- 
ments is there. 

In Iceland, PISA results have always received much attention in the 
media. The reporting of the PISA results in Iceland had considerable sta- 
bility as the same person, Almar M. Halldórsson, was until recently the 
project manager of PISA and the main mediator of what was highlighted 
in the PISA results for Iceland. He authored or co-authored all the 
Icelandic state reports from the beginning of PISA in 2000 and until 
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2013 (Björnsson et al., 2004; Halldórsson, 2006; Halldórsson et al., 
2007, 2013, 2007, 2013). He has also written some academic journal 
articles on the Icelandic gender gap in PISA (Halldórsson & Ólafsson, 
2009; Ólafsson et al., 2006). When the Directorate of Education took 
over the PISA project, the institution started to authorize the OECD and 
the Directorate as the authors (Menntamálastofnun, 2017; 
Menntamálastofnun & OECD, 2019). The ILSA documents of TALIS 
kept its personal authorization by Ragnar F. Olafsson. This different gov- 
ernmental process of authoring reports from the same institution tells a 
story about Icelandic governance and its inconsistency. This process also 
mirrors the emphasis on the PISA results at the Directorate office led by 
the former head of the Education Department at the Ministry of 
Education Dr. Arnór Guðmundsson, that was appointed by Illugi 
Gunnarsson, Minister of Education 2013—2017, as the first director of 
the new institution after having led the editorial work of the White Paper 
2014 (WP2014) (Ministry of Education, Science and Culture, 2014a). 

Most of the time, there have been weak ties between the higher educa- 
tion field in Iceland and OECD data production for the education gov- 
ernment body, as revealed by the low percentage of academic references 
in the bibliographies of the policy papers (Magnúsdóttir & Jónasson, 
Chap. 6 in this volume). This situation changed substantially with a new 
policy adopted by the Directorate of Education. For PISA 2015 and 
2018, there was systematic cooperation among professors from the 
School of Education at the University of Iceland and specialists from the 
Directorate in analyzing the Icelandic results of PISA 2015 and introduc- 
ing in a public forum. In 2016 peer-reviewed special issue on PISA lit- 
eracy was published by the School of Education, University of Iceland.* 
Thus, in recent years, the academization of PISA in Iceland has become 
markedly more prevalent. 

The institutional arrangement of OECD data production has now 
been introduced and the next step is to analyze the types and numbers of 
OECD references in the bibliometric database. Second, we scrutinize the 
authorship and the use of references in two OECD country case reports 
(shaded in Table 11.4) focused on the compulsory education in Iceland 
that were referred to in White Paper 2014 (WP2014). The analysis of the 
two case study reports gives some further insight to how the national 
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Table 11.4 OECD documents in the Icelandic database 


Year of Number of 

Document title in categories publication citations 

Policy reviews—Country case reports on Iceland 

Reviews of Vocational Education and Training—A 2013 1 
Skills Beyond School Commentary on Iceland 

OECD Study on Digital Learning Resources as 2008 1 
Systemic Innovation: Country Case Study Report 
on Iceland 

OECD Economic Surveys: Iceland 2013 2013 1 

OECD-Iceland Improving Schools Review. 2012 1 
Preventing Dropout in Upper Secondary 
Education in Iceland 

How's Life in Iceland? October 2015. OECD Better — 2015 1 
Life Initiative 

Education Policy Outlook: Iceland 2016 1 

Policy reviews—Other types 

The Definition and Selection of Key Competencies 2005 1 
Executive Summary 

Germany: Once Weak International Standing 2011 1 
Prompts Strong Nationwide Reforms for Rapid 
Improvement 

PISA 2012 Results in Focus: What 15-Year-Olds Know 2012 1 
and What They Can Do with What They Know 

Attracting, Developing and Retaining Effective 2005 1 
Teachers—Final Report: Teachers Matter 

Teaching Practices and Pedagogical Innovation. 2012 1 
Evidence from TALIS 

OECD Review of Policies to Improve the 2013 1 
Effectiveness of Resource Use in Schools (School 
Resources Review): Guidelines for Country 
Background Reports 

Global progress reports 

Education at a Glance 2013: OECD Indicators 2013 2 

Equity and Quality in Education: Supporting 2012 1 
Disadvantaged Students and Schools. Iceland— 
Country Note—Education at a Glance 2014: OECD 
Indicators 

International large-scale assessments (ILSAs) 

OECD: PISA Surveys 2000-2012— Educational 2012 1 
Testing Institute 

Skills Outlook 2013: First Results from the Survey of 2013 1 
Adult Skills 


(continued) 
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Table 11.4 (continued) 


Year of Number of 
Document title in categories publication citations 
Programme for International Student Assessment 2016 1 
First Results from the Survey of Adult Skills 2013 1 


Note: The first two reports (shaded area) are the ones that are further analyzed 
in this section 


institutions are working with the OECD when producing some national 
evaluation and policy data. 

In general, Iceland has few publications or documents that can count 
as green or white papers, and few of the documents that exist have cita- 
tions and reference lists. There is no tradition for reference lists when 
framing education acts or curriculum guides. However, the protocol is 
changing, and the most recent documents published by the state do have 
citations and reference lists. Only three documents published in the 
2013—2018 reform period fulfilled all requirements for this study. These 
documents were the only white paper (Ministry of Education, Science 
and Culture, 20142), published specifically as such (WP2014) and two 
green papers (European Agency for Inclusive Education, 2017; Ministry 
of Education Science and Culture, 2014b) that had a proper reference list 
for bibliometric analysis. The latter green paper (European Agency for 
Inclusive Education, 2017) was updated to count as a white paper 
(WP2017) soon after it was published (Magnúsdóttir & Jónasson, 
Chap. 6 in this volume). These three documents contain a total of 203 
references, more than half of which appear in the only green paper 
(GP2014) that is heavily referenced by domestic publishers (Statistics 
Iceland and the state) (Fig. 11.2). 

Combining the bibliographic information for these three documents 
reveals that the OECD is the third most cited publisher and the only one 
that is international. The Icelandic references are overwhelmingly govern- 
mental and statistical. As discussed in the national chapter, neither local 
nor global academia plays a big part in providing knowledge in these 
documents, according to the bibliography. Of external knowledge pro- 
viders the OECD is the most cited. The main reason for the frequency of 
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MINISTRY OF STATISTICS OECD ICELANDIC MINISTRY OF UNIVERSITY OF 
EDUCATION, ICELAND ASSOCIATION FINANCE AND ICELAND 
SCIENCE AND OF LOCAL ECONOMIC 

CULTURE AUTHORITIES AFFAIRS 


Fig. 11.2 Most cited publishers 


OECD references is the many citations in WP2014. Table 11.4 lists all 
the OECD publications included as references in these three documents. 

The green paper from 2014 was a country background report written 
as an input to the OECD Review of Policies to Improve the Effectiveness of 
Resource Use in Schools. The document was prepared in response to guide- 
lines the OECD provided to all countries. The white paper from 2017 
was written by the European Agency of Special Needs, so WP2014 is the 
only document that was written originally in Icelandic by officials at the 
Ministry of Education without any kind of "external help or guidance" 
(Magnusdottir & Jonasson, Chap. 6 in this volume). Of these OECD 
publications in the Icelandic database, 7896 are cited in the white paper 
from 2014. Analyzing WP2014 qualitatively through Atlas.ti software 
reveals that the OECD is much more prevalent than its reference list 
accounts for. The reference list has 36 citations when including in-text 
citations, 12 of which originated from the OECD. Through word count- 
ing, we determined that the OECD is mentioned 66 times in the 
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document (English version) on 21 of the 45 pages of texts (excluding the 
reference list). OECD citations in WP2014 are a mix of values, concep- 
tual framework, numerical data (mainly PISA), and advice. Still, numeri- 
cal data are the dominant form of knowledge that leads the advice given 
for Icelandic policy in the WP2014. 

By exploring the two country case reports (the ones shaded in 
Table 11.4) one can better understand the knowledge production and 
procedure. The working procedure is to hire Icelandic scholars or special- 
ists by the Ministry to write a background report that typically counts for 
a majority of the final country case report. These background reports are 
not published. That partly explains the scarce of published green papers 
in Icelandic governance. The OECD Study on Digital Learning Resources 
in Iceland (2008b) was based on six case studies in the Icelandic educa- 
tion system. The appendix provides a list of 47 people who participated, 
27 of whom were women. None is authorized but only mentioned as 
participants in the knowledge process. A group of local experts partici- 
pates in informative meetings. Skülína Kjartansdóttir, a former school 
principal and currently an adjunct at the University of Iceland, wrote the 
other case study report under review; OECD Reviews of Vocational 
Education and Training (2013). She was hired to the Ministry to work on 
this report and attended meetings with the OECD authors. Her back- 
ground report is not mentioned in the reference lists, though her name is 
mentioned in the acknowledgments along with some officials in the 
Ministry. There is no list of participants available in the report. References 
are 35, thereof 17 of which were published by the OECD. It is very simi- 
larto WP2014 in terms of heavy use of references from OECD. Conversely, 
the OECD Study on Digital Learning Resources in Iceland (2008b) has 13 
references, 5 of which are academic journal articles with only 1 OECD 
reference which is more in line with GP2014, referencing mainly national 
knowledge providers rather than OECD documents. 
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Concluding Discussion 


All case country analyses reveal multiple layers in their OECD-related 
references. On the one hand, all policy documents—both green papers 
and white papers—tend to follow the demands and credos of evidence- 
based policy. Perhaps more importantly, the documents associated with 
OECD policy instruments carry more weight than what can be seen in 
the mere list of references. First, this finding reinforces the fact discussed 
previously in this volume that a bibliometric reference is more than a 
reference and signals commitment in addition to relaying information. 
Second, following the theoretical framework of this chapter, this finding 
leads us to conclude that the references are shaped by power relations and 
in the sense of political capital. 

Drawing on all three case analyses, we find support for a hypothesis 
that the power networks that have been formed transnationally are mani- 
fest in the use of references in the documents analyzed. The same Finnish 
network of knowledge brokers functioned in PISA data collection, 
national education data collection, and ministry-commissioned national 
data gathering. In Iceland, the weak ties between national and interna- 
tional organizations formed a base of knowledge selection and use. This 
could be seen as a case where social capital is transferred into knowledge. 
In the Danish case, we also see the clear contours of a powerful national 
network of protagonists associated with evidence-based policy advice in 
general and the results from OECD reports and ILSA data in particular 
that have been able to exert considerable influence. The basis of this influ- 
ence is found both in vocal media appearances and in participation in the 
relevant government bodies and consortia. 

A mere quantitative analysis of references would seem to suggest that 
the OECD plays only a minor role in national education policies due to 
the fact that its references are in the minority despite being an important 
international reference in the analyzed country cases. However, it is our 
understanding that the importance of the OECD was not best visible in 
these references. When deeply analyzing each of the cases, we identified 
an appreciation of knowledge and an increased importance of national 
institutions through links to the OECD. This observation raises serious 
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questions for bibliometric analysis and complex questions for further 
analysis and conclusions. 

Without the pre-existing knowledge from other studies on the OECD, 
the quantitative analysis would probably have erred toward assigning the 
OECD a less important role than it has. This finding remains a warning 
for further analysis. From another perspective, one could argue that this 
analysis now gives the OECD a more important role than it deserves, as 
we have dug out different networks and connections with the 
OECD. Some perspective as to whether this argument stands could be 
drawn from an attempt to understand what would have been the alterna- 
tive results. For instance, is it coincidental that the Finnish universities 
with more connections to the OECD are more referenced than the ones 
without? At the very least, the power of knowledge seems to channel 
through the same hubs. The OECD is indeed powerful in forming an 
epistemic community and a constituency underpinning its policy instru- 
ments that is more powerful than other international organizations in the 
Nordic countries. These findings correspond well with Grek's aforemen- 
tioned argument about the OECD taking up a position as a boundary 
organization in the Nordic region. Further studies should now move to 
unravel the configurations and workings of the nexus or assemblage that 
creates the basis for this power. 


Notes 


1. Between 1961 and 1964, Eide headed the OECD's work on education 
planning, and he served as the first board chairman of the OECD Centre 
for Educational Research and Innovation (CERI) after the formal estab- 
lishment in 1970. He continued his affiliation with the OECD education 
organization throughout the 1980s. 

2. https://www.oecd.org/finland/publicationsdocuments/reports/ 

3. In addition, there were mentions of student interviews and interviews 
with the Central Board for Education in Helsinki. 

4. See, for example, this special issue on PISA 2018 in Icelandic http://netla. 
hi.is/?page_id=4720 and in 2016, especially about literacy: http://netla. 
hi.is/serrit/2016/um laesi/04 16 laesi.pdf. 
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Regional Policy Spaces, Knowledge 
Networks, and the “Nordic Other” 


Saija Volmari, Kirsten Sivesind, and Jon Torfi Jonasson 


Two core beliefs that have helped promote evidence-based policymaking 
in education are that school knowledge is abstract and universal and that 
“empirical evidence is an efficient indicator of knowledge and learning” 
(Wiseman, 2010, p. 1). In fact, these beliefs have popularized the what- 
went-right or best practices approach in education policy planning. The 
what-went-right approach implies that it is possible to empirically mea- 
sure students’ learning in ways that produce generalized knowledge and 
that these transferrable features enable policymakers and others to apply 
findings across a variety of contexts. This view on the transfer of 
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knowledge partly explains why, for example, the Programme for 
International Student Assessment (PISA) has developed into a global 
project, supported by a large number of stakeholders (Addey et al., 2017). 

However, previous research has questioned the smooth transition of 
empirical evidence across countries and continents as well as the universal 
character of the findings and recommendations produced by interna- 
tional organizations (Bieber, 2016). First, the receptiveness to interna- 
tional comparisons and the uptake of global scripts and schemes in 
policymaking depends on national and local traditions and cultures. 
Steiner-Khamsi et al. (2020) demonstrated that policymakers under the 
jurisdiction of the state are selective in their application of evidence from 
international large-scale studies. Second, for the type of transfer associ- 
ated with PISA and similar studies, it is a common belief that policy 
transfer occurs and works best when educational systems are similar in 
terms of the challenges they are facing (Steiner-Khamsi, 2013, pp. 20-21). 
Moreover, it is assumed to be more likely that neighboring countries will 
be more receptive to the same solutions on common problems, than 
countries in other regions or continents. Based on these arguments, 
neighboring countries select the same policy solutions since they are 
committed to the same culture or have established similar systems and 
models to warrant legal rights on behalf of the citizens. For this reason, 
neighboring states may reflect similar temporal and geographic patterns 
of policy spread (Dolowitz, 2018). 

However, to date, researchers have reached no consensus about the role 
that regional contexts play in transnational policy transfer and the way 
this context shapes the receptiveness to evidence of what counts as rele- 
vant knowledge in school reform policy. In fact, Dolowitz (2018) reported 
on the criticism that diffusion studies do not capture the cultural and 
social complexity of policy transfers within the global-local policy nexus. 
Also Steiner-Khamsi (2013) refers to policy borrowing and lending that 
takes place despite cultural differences, simply because policy transfer 
does not necessarily adhere to a rational logic or pattern. Therefore it is 
important to apply various models and analytical lenses to study policy 
transfer in a regional context. Thus, this chapter employs various data 
sources and methods to examine how policymakers and experts in three 
Nordic countries locate themselves in a larger political reference space 
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when they develop ideas and collect facts and evidence to justify school 
reforms in their respective countries. The following questions drive 
this study: 


— How and why do the experts and policymakers communicate with 
Nordic colleagues about the significance of various issues and ideas? 

— Do government-appointed expert panels and policymakers within 
state administrations consider knowledge sources from other Nordic 
countries relevant? 

— Moreover, do they share a specific interest in authorizing such sources 
by referencing them? Why? Why not? 


By examining the latest (curriculum) reforms in three Nordic coun- 
tries (Finland, Iceland, and Norway) in more detail, this chapter attempts 
to identify the actual role and influence of regional cooperation in rela- 
tion to education reform and the status and use of regional references. 
The chapter also aims to contribute to the discussion on the role of 
regional context in transferring and translating transnational policy 
knowledge within the national context of policymaking processes. 


Policy Borrowing Within and Beyond 
Regional Contexts 


Nordic countries are regularly seen, from the outside but also within 
these countries, as one cultural and political territory. They are believed 
to share the same value base of the Nordic welfare state related to democ- 
racy, equality, and social justice along with a commitment to welfare poli- 
cies that strive for societal well-being for the whole population (Arnesen 
et al., 2014; Delhey & Newton, 2005; Jacobsson et al., 2004). These 
countries also share a common history, as they have at times had the same 
head of state and even formed one union or state with varied composi- 
tions (Berntzen, 2017). 'Ihese political traditions are also evident today. 
The Nordic countries have all developed political systems in which the 
nation-state provides welfare services and largely regulates the public 
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sector. Due to the central role of state authority in public governing, they 
have developed systems to take public interests into account. Hearings 
and public inquiries symbolize this kind of system. 

In the field of education, the public school systems likewise share con- 
siderable similarities, often referred to as "the Nordic model in educa- 
tion” (Telhaug et al., 2006). The comprehensive basic education in all five 
countries consists of 9-10 years of compulsory education. There is an 
uncontested unanimity in the region that comprehensive basic education 
should be free of charge. Education policies across the Nordic countries 
have all been strongly influenced by social-inclusive aspects. Despite 
recent tendencies to introduce more market-driven policy solutions, edu- 
cation in the Nordic region is still seen as a crucial instrument for increas- 
ing social justice in all Nordic countries. (Arnesen & Lundahl, 2006; 
Lundahl et al., 2018; Lundahl, 2016) 

Considering these presumable common denominators of the Nordic 
countries, one may expect that there exists a Nordic space for policy 
cooperation and thereby a considerable policy transfer of knowledge 
within the region. However, three particular aspects must be taken into 
consideration in attempts to identify this space. First, as the political sci- 
entist Paul Cairney (2016) has suggested, to understand the use of evi- 
dence in policymaking processes, one needs to understand both how the 
policymaking process works (p. 10) and how policymakers fit into it 
(p. 6). Despite their common history and similar political traditions, the 
decision-making procedures and the role of different policymakers in 
them may well vary. Hence, it is crucial to take both similarities and dif- 
ferences into account and not to assume that these countries are necessar- 
ily uniform in terms of political decision-making processes. Second, since 
the 1990s, the traditional Nordic values of universal well-being and edu- 
cation have been increasingly influenced by political interests in neolib- 
eral values, individualization, marketization, and technologies of new 
public management. However, differences exist in the extent to which 
this interest has turned into a dominating feature and influenced educa- 
tion policy and reforms in various Nordic countries (Arnesen et al., 2014, 
p. D). Third, national reforms, policies, and policy discourses have become 
globally framed and are partly influenced by international and suprana- 
tional organizations, such as the Organisation for Economic Co-operation 
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and Development (OECD) and the World Bank (Rizvi & Lingard, 
2010). These organizations have become active in producing large-scale 
comparative data and thus involved in advocating education reform 
agendas in the Nordic region. There are good reasons to consider the 
influence of these organizations and their programs in national reform 
making and national policies as contingent (Sivesind & Wahlstróm, 2016). 

For instance, references to the OECD are partly related to how the 
data match central topics and issues in national reform processes. 
Moreover, although the OECD is a politically influential organization, 
the use of its data is mostly detached from its ideological views. In par- 
ticular, numeric and comparative data from such organizations are used 
as evidence. Numbers are convenient tools in the current culture of 
evidence-based policymaking, as they appear to be neutral, apolitical, and 
objective (Stone, 2016). They represent what Gil Eyal (2019) called 
"mechanical objectivity" (p. 115) in policymaking, that is, a legitimation 
strategy pursued by referencing, rankings, and quantified comparisons 
(see discussion in Chap. 2). 

Previous research has discussed the regional education space particu- 
larly from the point of view of the European Union influence and estab- 
lished the term “European education space” (see, for instance, Lawn 
et al., 2011; Grek & Rinne, 2011). Researchers have demonstrated that 
this policy space has been extensively shaped and sustained by "governing 
by numbers" (Ozga, 2009; Rose, 1991) as a technology of governing. 
Grek and Rinne (2011, pp. 29—30) and Martin Lawn et al. (2011) 
claimed that this policy space has been constructed through harmoniza- 
tion attempts, soft governance, benchmarking, and comparison, which 
evolved after governments agreed upon the Lisbon treaty (Grek & Rinne, 
2011). In addition to law and regulations, transnational expert networks 
play a significant role in how this process is evolving (Lawn et al., 2011). 
In a recent article, Grek et al. (2020) characterized this form of regional 
space as resulting from how experts “make meaning and attempt coher- 
ence in networked forms" (p. 4). One concern in these contexts is to 
achieve consensus about what to measure in terms of large-scale 
assessment. 

However, comparisons and numeric data produced by the OECD and 
similar organizations are not by themselves a sufficient explanation for 
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how regional spaces evolve. As Espeland (2015) has underscored, the 
objectiveness of numbers and their generic character are appealing since 
nearly any political narrative can easily be attached to them. In other 
words, the generic character of numbers supports the creation of political 
narratives. Thus, the narratives of the regional practices and mindsets can 
serve as a lens through which to understand cooperation and shared 
interests that shape Nordic education policy space in particular ways. 

Our hypothesis is that this regional Nordic policy space, although 
partly gaining its significance from international networks, operates dif- 
ferently than for instance the European policy space. One reason is that 
the Nordic region by itself does not produce numbers for assessing educa- 
tion, like the OECD does. Thus, the Nordic community of policymakers 
and experts relies on something other than agreements upon benchmarks 
and standards, and we set out to explore this Nordic “other.” This chapter 
will examine how this "other" is constructed by relating to a shared policy 
space that is developed and sustained in various ways. 

To analyze Nordic cooperation as a regional space, we will draw on 
Massey's (1994, p. 2) conceptualization of spatial as “the social stretched 
out” and her idea of space as not fixed and static but rather “a product of 
interrelations and constituted through interactions" (p. 7). Following this 
idea, we look at the Nordic policy space not as a separate layer between 
the national and transnational, but as a space where national actors inter- 
act and form connections to make sense of and give meanings to the 
international. Thereby, they construct a regional mindset, which we refer 
to as the Nordic "other." As such, the Nordic policy space is in a state of 
constant becoming and fluidity, created and re-created through the inter- 
actions of the involved policymakers, experts, and stakeholders and their 
narratives of relating to each other. 


Country Cases 


As we have seen in the previous country chapters, the number of refer- 
ences to other Nordic countries (i.e., regional references) identified 
through bibliometric network analysis was surprisingly low. We also 
found that the country distribution of regional references varied to some 
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degree. Norway used 66 regional references, significantly more than 
Iceland, where only 4 references were regional. Among those regional 
references used in Iceland source documents were a World Health 
Organization (WHO) document published in Denmark and documents 
produced by the European Agency of Special Education, placed in 
Denmark. In Finland, we identified 11 references as regional. The rare 
use of regional references and the frequent use of national references 
prompted us to wonder why this pattern had become so biased. 

In order to illustrate the regional network pattern of references, we 
conducted a bibliometric network analysis, which we present in more 
detail later in this chapter. We also decided to combine this network 
analysis with complementary data sources to examine the three country 
cases—Finland, Iceland, and Norway—in more detail. These three coun- 
tries are similar in terms of their legacies as young nation-states compared 
to Sweden and Denmark (Elgenius, 2011), but they differ in their 
decision-making procedures and in the role that policy documents play 
in these processes. Thus, we decided to conduct interviews with experts 
with knowledge and information about these features of the system. All 
experts interviewed were involved in the national reform processes that 
are examined in this book. 

We should also mention that the reforms we examined were partly dif- 
ferent in terms of their timing, focus, and scope. In the case of Finland, 
we examined the latest reform of the National Core Curriculum (2014). 
As explained in Chap. 5 in this volume (Volmari, Kauko, Anturaniemi, 
& Santos), in the case of Finnish curriculum reform, the research team 
made the decision to focus on the reform of the objectives of the National 
Core Curriculum and the distribution of lesson hours from 2012 
(Government Decree 422/2012). The government decree is based on a 
white paper and it delegates the power to decide on the content of the 
core curriculum to the National Agency for Education, within the frame- 
work the government decree sets for this work. The most prominent 
policy documents in this process were the white paper Future Basic 
Education from 2012 (Opetus- ja kulttuuriministerió, 2012) and the 
green paper Basic Education 2020: Common National Aims and Division 
of Teaching Hours from 2010 (Opetus- ja kulttuuriministerió, 2010). This 
green paper was originally designed as a white paper, but it was discarded 
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at the last minute due to political disagreements and a change in political 
power. Nevertheless, the 2012 white paper was mainly based on the green 
paper preceding it, as demonstrated in Chap. 5. Both working groups 
utilized expert hearings and statements and extensively involved different 
stakeholder groups, particularly in the process of the green paper. For 
example, the working group of the green paper 2010 received commen- 
tary from over 200 municipalities in Finland and utilized a survey of over 
60,000 students on their experiences in school and their vision of an 
ideal school. 

In the case of Iceland, we base our analysis in this chapter on three 
documents used to govern Icelandic compulsory education, together 
with the law, regulations, and formal curriculum (see Chap. 6). Two of 
these are classified as white papers and one as a green paper. As discussed 
previously in Chap. 2, white papers are political by default and therefore 
influenced by the government and the minister in charge of the policy 
processes. However, the political influence by the administration of the 
same government varies depending on the continuation of policies over 
time across various governments. If the background work initiated by 
one minister leads to the continuation of policy by the previous minister, 
then the background work sets the agenda and may not even reach the 
status of being enquired by a green paper. This is often the case in Iceland, 
where recent decades have offered many examples of extensive back- 
ground work being conducted and used to underpin reform bills and 
resolutions in the form of extensive discussion and consultation rather 
than in the form of public enquiring and formal written documents, 
which we refer to as green papers. Thus, although the working mode of 
Icelandic policymaking relies, sometimes extensively, on internal and 
external documentation, this evidence does not necessarily reach the 
stage of reference lists or other formal acknowledgment. 

This situation varies starkly from the Norwegian case. As already pre- 
sented (see Chaps. 7, 9, and 10), Norway has expended a huge amount 
of resources on public commissions with a mandate to evaluate and make 
recommendations to decision-makers regarding how to reform and renew 
the public sector. The government appoints members to sit in these com- 
missions. In this chapter, we refer to eight such reports and two white 
papers. We include references in these documents in our bibliometric 
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network analysis, while two particular inquiries (Bostad and Ludvigsen 
commissions) and one white paper (Subjects—In-depth Learning— 
Understanding, see: Appendix 1) serve as the main data source for our 
qualitative study. While the Bostad commission made inquiries and 
advised the ministry and parliament on how to revise the education 
clause formulated within the School Act, the main work task of the 
Ludvigsen commission was to recommend overall goals, aims, and struc- 
tures for renewing the national curriculum. A group of officials with 
educational-scientific backgrounds and experience from the Directorate 
of Education assisted the experts during the writing process, and civil 
servants within the Ministry of Education were the main actors in the 
process of formulating the white paper. They worked closely with the 
political leadership to finalize the paper. Throughout the process, various 
groups of stakeholders were invited to provide comments. 

Altogether, the three cases selected for this study vary in terms of orga- 
nizational arrangements for preparing political decision-making pro- 
cesses. For instance, in contrast to Norway, academic Participation in 
Finnish preparatory working groups has declined during the last decade. 
In the national core curriculum reform of 2010-2016, the working 
groups we examine in this chapter contained no scientists, although sci- 
entific experts offered their insights. The members of the working group 
of the white paper from 2012 (Source Document 66 in the Finnish sam- 
ple, see Chap. 5) consisted only of the civil servants of the Ministry of 
Education and Culture. In the working group of the green paper, mem- 
bers came from the main political parties and stakeholder organizations 
in addition to civil servants. There were no representatives from other 
countries. Norway chose a rather opposite approach. A scientist led 
Ludvigsen’s committee, which included other scientist members as well. 
Two of the members represented other Nordic countries, namely 
Denmark and Sweden. Conversely, Iceland has adopted no formal proce- 
dures to produce white papers based on inquiry bodies, although the 
government plans to move in this direction. It is up to each minister to 
determine how the work is approached, the amount of party political 
involvement, and the influence of stakeholders or various professional 
groups. Parliament is not involved until a very late stage in the formula- 
tion of laws within education. To conclude, the three case studies varied 
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in their timing, focus, and scope, as well as in terms of organizational 
arrangements. But they also varied in what was valued as expertise and 
how and in which stage of the reform process expertise was utilized. 


Data and Methods 


Our research design originally stemmed from the five national country 
cases. Across the five Nordic countries, there was one finding that par- 
ticularly puzzled the research teams. In spite of what might be expected 
and what previous research on education transfer has predicted, we found 
no solid evidence of regional policy transfer or learning. In fact, the num- 
ber of regional references in each country was incredibly low, and a shared 
knowledge base was almost absent. This sparked our research question 
and design, and we decided to combine methods to see if we could find 
any evidence of Nordic policy learning and sharing that were not neces- 
sarily authorized within bibliographies and footnotes. 

To begin, we conducted a bibliometric network analysis of the regional/ 
Nordic references to identify any evidence of a common Nordic knowl- 
edge base. In the bibliometric network analysis, we used the software 
program UCINET to generate statistics on Nordic references and draw 
illustrations of the Nordic knowledge network. The method of biblio- 
metric analysis and how it was applied in this research project is explained 
in more detail in Chaps. 1 and 2. However, this method did not appear 
to offer a complete picture of Nordic policy cooperation and of the pos- 
sible knowledge exchange within the Nordic region. 

Inspired by Tórnberg and Tórnberg (2019), we perceive networks as 
cultural products in which *discursive and cultural elements play out 
and shape the networks in which they exist” (pp. 61-62) and adopt 
their suggestion to complement network analysis with other methods, 
namely with expert interviews. We conducted interviews with members 
of the key working groups and committees in Finland and Norway and 
some key ministerial administrators in Iceland. We used the interviews 
both to obtain new insights into regional knowledge sources and to 
acquire knowledge about Nordic cooperation and knowledge-sharing, 
which cannot be disentangled by bibliometric network or document 
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analysis. We used thematic expert interviews with the same list of topics 
in all three countries. We conducted 18 semi-structured interviews in 
total, 8 in Norway, 5 in Finland, and 5 in Iceland. We recorded some 
interviews using Zoom and others with single recorders in physical set- 
tings, and all interviews were transcribed to ensure the validity of our 
approach. 

We analyzed the interview data while keeping the following points in 
mind. First, informants were experts in their field and had been members 
of the preparatory committees or working groups in the reforms we 
examined. Hence, the information they provided can be seen as reliable 
firsthand information. However, we acknowledge that the informants 
shared with us their accounts and recollections, sometimes of the work 
they were involved in almost ten years ago. Attentive to these two vantage 
points, we decided to categorize our interview findings into clusters of 
main narratives. 


Results 


In our study, we set out to identify the actual role and influence of the 
regional, Nordic cooperation in relation to our three country examples 
(Finland, Iceland, and Norway) of an education reform, and the status 
and use of regional, Nordic references in these reforms. Our main meth- 
ods were bibliometric network analysis and thematic expert interviews. 
We present our findings below, divided into two subsections according to 
our two main methods. 


Bibliometric, Regional References, and the Absent 
Nordic Other 


The bibliometric analysis reveals, in part, where different countries have 
looked for inspiration and influence. The whole database, including the 
references of all five countries, had a total of 225 regional references, 
meaning references where a country referred to a publication published 
in another Nordic country. There are numerous reasons to expect that 
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one could detect notable policy referencing of information sources 
between the Nordic countries in our data. However, this does not seem 
to be the case. Ihe number of Nordic references used in these reforms was 
surprisingly low compared to the use of domestic and international refer- 
ences (see Baek, Tiplic, and Santos, Chap. 9, Table 9.3). References from 
other Nordic countries amounted to approximately 2—796 in all of the 
countries among the five cases, which we consider as a key finding. 

In Finland the regional Nordic references amounted only to 1.6396 
and in Iceland 2.0896. In Norway, 7.0996 of all the references were from 
Nordic sources, but compared to the amount of domestic (66.8396) and 
international references (26.0896), this percentage was low as well. 

In Table 12.1, we can see how references to sources published in 
another Nordic country were distributed between the five countries. 
Altogether 44% of the time, the referenced documents were published in 
Sweden, while 3196 of the time, a country referred to a publication from 
Denmark. Sources published in Finland or Norway were referenced less 
than 1096 of the time and Icelandic sources only 196 of the time. Based 
on Table 12.1, one can conclude that the regional evidence base consists 
predominantly of evidence published in Denmark or Sweden. 

Table 12.1 also shows how the regional references were distributed in 
each country. Documents published in Sweden were the most referenced 
in Denmark, Finland, and Norway. For example, five out of eight Nordic 
references made in the Danish source documents were published in 
Sweden. In the Finnish source documents, 7 out of 11 cited Nordic refer- 
ences were of Swedish origin. In Norway, the 180 regional references 
cited in the source documents were distributed slightly more evenly 
between Denmark (70) and Sweden (77). Iceland served as an exception 
to this pattern, as in the Icelandic documents only four regional docu- 
ments were cited, all of which were published in Denmark. However, one 
was produced by an international agency and three were by a European 
agency. In the case of the Swedish reform, 4496 (8 out of 18) of the 
regional references cited in the source documents were published in 
Denmark, with 4 from Finland and 5 from Norway. 

Although Danish and Swedish documents were used more than the 
Finnish ones, the most prominent documents in the Nordic knowledge 
network would appear to be those used more than once and in more than 
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one Nordic reform. When we look at co-citations, or those sources refer- 
enced by more than one document, the pattern tells a slightly different 
story. Out of the seven most co-cited sources, three were published in 
Denmark, two in Finland, and two in Sweden. The Danish and Swedish 
publications were cited in only the Norwegian policy documents. Both of 
the Finnish publications were cited by two different Nordic countries. 
Specifically, the Finnish National Core Curriculum of 2014 was cited 
two times by different Norwegian sources and one time by a Swedish 
source. Likewise, the Finnish National Core Curriculum of 2004 was 
cited two times by different Norwegian sources and once by a 
Danish source. 

Moreover, the two most cited Finnish documents were actual national 
core curricula for comprehensive education (2004 and 2014), of which 
we examine the 2014 curriculum as an example of a Finnish education 
reform. 'Ihese documents, the 2004 and 2014 National Core Curricula 
of Finland, were also the only ones among the most-cited regional docu- 
ments that were actual national core curricula referenced by more than 
one country. They were part of the knowledge base of the reforms in 
Denmark, Norway, and Sweden, which indicates that other countries 
have looked upon Finnish curricula as prominent regional references. 

Figure 12.1 illustrates the whole network, further confirming the find- 
ings explained above. Norway, in white, clearly co-cited a cluster of 
regional knowledge most, as indicated by the arrows pointing outward, 
indicating policy borrowing. The network consists of predominantly 
Danish (red quadrants) and Swedish (yellow quadrants) references, which 
were used in the majority of Norwegian source documents (white dots). 
The two Finnish (blue quadrants) documents are medium-sized, as they 
were referenced less by other Nordic countries than the most co-cited 
documents of Denmark and Sweden, which appeared in the 
Norwegian data. 

As these results demonstrate, the regional Nordic references were 
mostly absent in these five reforms since there were relatively few co- 
citations. National chapters have explained this as a matter of self- 
referencing (Chaps. 5 and 8). However, Norway was a small exception 
among these countries, with its regional references amounting to almost 
7% (see Table 12.1) and with a cluster of references that were co-cited 
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Fig. 12.1 Network of the Nordic references. (Note: This figure includes only ref- 
erences for which the publication location differed from the publication location 
of the source documents. Denmark = red; Finland = blue; Iceland = orange; 
Norway = white; Sweden = yellow; unclear Nordic = black) 


(see Table 12.2). By looking into these co-cited references, we find that 
the reform made use of some reports published in other Nordic countries 
more actively than the level observed in other Nordic countries docu- 
ments. Yet, even in Norway, the number of regional references was lower 
than originally expected on the basis of previous theories of policy trans- 
fer within a region and between similar cultural contexts. 


Cooperation and Communication Within the Nordic 
Policy Space 


The Nordic countries maintain close political and policymaking connec- 
tions to each other, and their governments have formal connections at 
multiple levels, for example, through the work of the Nordic Council 
(founded in 1952) and the Nordic Council of Ministers (founded in 
1971). From 1967 national ministries in education initiated Nordic 
research collaboration to stimulate more advanced use of teaching 
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Table 12.2 Most co-cited Nordic references 


Location of 

ID Title publication Type In-degree 

5781967 Laererkompetanse og elevers Denmark 1 7 (cited by 7 
leering i barnehage og skole. Et different 
systematisk review utført for Norwegian 
Kunnskapsdepartementet, Oslo. sources) 
København: Danmarks 
Paedagogiske Universitetsforlag 
og Dansk Clearinghouse for 
Uddanelsesforskning [Teacher 
Competence and Students' 

Learning in Preschool and 
Primary Schooling. A Systematic 
Review Conducted for the 
Ministry of Education and 
Research, Oslo] 

5781631 Utmárkt undervisning: Sweden 2 4 (cited by 4 
Framgángsfaktorer i svensk och different 
internationell belysning. Norwegian 
Stockholm: Natur och kultur sources) 
[Excellent Teaching: Factors in 
Making Progress Within 
Swedish and International 
Perspectives] 

5782251 Betydelsen av icke-kognitiva Sweden 4 4 (cited by 4 
fórmágor. Forskning m.m. om different 
individuella faktorer bakom Norwegian 
framgáng. Sverige: Skolverket sources) 
[The Significance of Non- 
cognitive Skills. Research About 
Individual Factors That Explain 
Improvement] 

5782322 Grunderna för läroplanen för den Finland 4 3 (cited by 2 
grundläggande utbildningen different 
2014 [The Curriculum for Basic Norwegian 
Education 2014] sources and 

1 Swedish 
source) 


(continued) 
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Table 12.2 (continued) 


Location of 
ID Title publication Type In-degree 
5782324 Grunderna för läroplanen för den Finland 4 3 (cited by 1 
grundläggande utbildningen Danish 
2004 [The Curriculum for Basic source and 2 
Education 2004] different 
Norwegian 
sources) 
5781048 Basic learning: Thematic report 1: Denmark 1 2 (cited by 2 
Ensuring basic skills for all, different 
Working Group 1, København Norwegian 
sources) 
5781956 Resultater fra Denmark 1 2 (cited by 2 
Kartleggingsundersøkelse i different 
Kristiansand kommune 2013 Norwegian 
[Results from a Survey (on sources) 


students’ learning performance) 
in Kristiansand] 


Note: “Nordic” references refer to those published in a Nordic country that are 
cited by source documents published in a different Nordic country (e.g., Danish 
documents cited by Swedish source documents or Finnish documents cited by 
Norwegian source documents) 


technology. From early on, these Nordic institutions initiated evalua- 
tions, conferences, and formal inquiries to address issues of significance 
for the governance of Nordic school reforms. One known report pub- 
lished by the Nordic committee for educational research, chaired by Johs 
Sandven, addressed, for example, how programmed teaching and innova- 
tive usage of new technology can improve teaching in schools (Dahllöf & 
Wallin, 1969). Another report about the administration of the Nordic 
school systems was published in 1974 after two years of work by a Nordic 
working group of civil servants who had the mandate to help harmonize 
the national curricula for the compulsory school systems (Nordiska rådet, 
1974). They were also asked to provide information about the Nordic 
school system of relevance for teachers, associations, and administrative 
bodies. The Nordic Council (Nordiska Ministerrådet, 1978) made a revi- 
sion of the report four years later. Here, they presented formal 
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information about the different school types within the education system 
and how they were reformed within the single country. 

Later reports offered evidence of systematic cooperation across the 
Nordic countries, partly formalized through a governing body of civil 
servants from the Nordic ministries and a working group that was respon- 
sible for meetings and conferences (Nordisk Ministerrád, 1990). One 
outcome in the early 2000s was the conference report Vision and Reality 
(Sigurdardottir & Harðardóttir, 2000), which addressed evaluation and 
assessment in Nordic schools. A later report from the Nordic Council of 
Ministers was Northern Lights on PISA 2003—A Reflection from the Nordic 
Countries (Mejding & Roe, 2006). 

This report, however, did not originate from the work of the Nordic 
Council; instead, it was written by Nordic researchers who collaborated 
within the OECD. The Department of Teacher Education and School 
Development at the University of Oslo had published an earlier report 
entitled Northern Lights on PISA, Unity and Diversity in the Nordic 
Countries in PISA 2000 (Roe et al., 2003). This report stated that PISA 
represented a new commitment by the governments of OECD coun- 
tries to monitor the outcomes of education systems with a focus on 
learning achievement. From then on, it seems that Nordic data and 
knowledge about the quality of the school systems was hijacked by 
organizations other than the Nordic Council. Yet, meetings among civil 
servants and experts within the context of the Nordic Council 
continued. 

To further illustrate the frequency of Nordic cooperation in the field of 
education, Table 12.3 presents the number of meetings taking place 
between top political and administrative levels, as a matter of course, 
every year within the arena of education and research (and a similar pat- 
tern would be obtained for a number of other arenas). 

The informants in all three countries talked about constant and regu- 
lar formal cooperation, such as annual meetings of the directors of the 
national education agencies, meetings of Nordic curriculum specialists 
once or twice a year, and cooperation within the Nordic Council of the 
Ministers. In addition, ad hoc groups are formed to deal with pressing 
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Table 12.3 Number of meetings each year 2013-2019 under the auspices of the 
Nordic Council and Nordic Council of Ministers 


Nordic Council for Culture Nordic Council of Ministers for Education 


and Education and Research 
Number of 
Number of meetings of ministerial Number of ministerial 
political representatives meetings administrator meetings 
2013 1 5 
2014 1 4 
2015 1 4 
2016 5 2 4 
2017 5 2 4 
2018 5 1 4 
2019 5 1 4 


Source: https://www.norden.org/en/organisation/nordic-co-operation 

Note: These meetings related to education involved at different times 
parliamentary representatives, ministers of education, and civil servants at the 
ministries of education. Adapted from the webpage Nordic cooperation, which 
describes the organization of both the Nordic Council and the Nordic Council of 
Ministers 


issues when the need arises. Although the bibliometric network analysis 
demonstrated a very low percentage of regional references, with Norway 
being the only Nordic country that seemed to refer substantially more 
to other Nordic countries, the expert interviews revealed active and 
vivid cooperation and communication among the Nordic countries. 
Our informants noted that, in addition to the official meetings, several 
groups met regularly at the Nordic ministerial level or among the gov- 
ernmental organizations outside the purview of the formal Nordic 
Council cooperation. Several experts talked about frequent but more 
informal communication and collaboration, like phone calls and study 
visits. Coherent documentation of this is not available as formal min- 
utes are not always written or centrally available even though all the 
meetings are filed within individual ministries. There is no question of 
extensive Nordic contact and discussion at the Nordic policy level, but 
little evidence is available about the actual impact of those meetings on 
policy formation. 
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Legitimation, Policy Dynamics, and the Omnipresent 
Nordic Other 


Since the bibliometric network analysis revealed a very low use of regional 
Nordic references despite well-established Nordic political and cultural 
cooperation, we decided to conduct interviews with the key experts 
involved in the reforms in question in Finland, Iceland, and Norway. In 
a total of 18 interviews, we asked the informants questions about their 
role in the reform or working group, their professional background, and 
the issues and themes they considered relevant during the reform process, 
in the work ofa working group or commission, or in the Nordic coopera- 
tion more generally. In addition, we asked about any national, Nordic, or 
international conferences, workshops, or meetings they attended during 
the process with a particular focus on any Nordic meetings or other forms 
of Nordic cooperation. 

We also specifically asked them for possible explanations for the lack of 
Nordic references in the key policy documents. The absence of regional 
bibliometric references in our data came as a surprise to most informants, 
as they all considered Nordic cooperation important and meaningful. 
From responses to the question on the possible reasons for the absence of 
references to Nordic knowledge sources in the actual policy documents, 
we identified four main narratives the informants used to explain this 
absence: 


1. Nordic cooperation is not well documented and does not produce 
data that lends itself easily to reference. 

2. Nordic cooperation is such an implicit part of policy cooperation that 
it does not need specific mentioning. 

3. Legitimation function and hierarchization of evidence play roles when 
choosing evidence to reference in written documents. 

4. In the curriculum reforms, national orientation and political dynam- 
ics still play the greatest role. 


The first narrative our informants provided for the absence of Nordic 
evidence in the actual written documents was that Nordic cooperation is 
often rather informal in nature and hence not well documented. Earlier 
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in this chapter we discussed the official forms of cooperation. 
However, many informants asserted that a significant part of the coopera- 
tion occurs in more informal contexts and ways. For instance, they 
described study visits as being very common between the Nordic coun- 
tries. These visits seemed to be particularly common when a country was 
in the middle of a reform process. Informants in all countries reported 
this kind of visit as one form of collecting knowledge and ideas. Several 
Finnish and Norwegian informants mentioned the study visit of the 
Norwegian Ludvigsen’s committee to Finland. Although the Norwegian 
committee prompted the visit and prepared a set of questions it specifi- 
cally wanted to discuss to learn about the recent Finnish core curriculum 
reform, the Finnish informants described the meeting as one of mutual 
exchange and learning. In general, the informants described Nordic edu- 
cation cooperation as the "kind of expert kind of cooperation, where we 
do discuss very profoundly why someone has done something" (working 
group member, Finland). Since this kind of cooperation is not docu- 
mented in memos, meeting minutes, or reports, it does not produce data 
that lends itself to reference in the sense of assertions backed up with 
information (Cairney, 2016) like the bibliometric references examined in 
this chapter. 

The second narrative we discovered is that Nordic cooperation is such 
an implicit part of policy cooperation that it does not need to be men- 
tioned specifically. This second narrative is closely linked to the first one. 
Most informants declared that in general other Nordic countries were 
always looked at and their policy developments carefully followed and 
that in any reform process, one tended to always start by looking at other 
Nordic countries. One informant specifically explained that ^we always 
start by looking at other Nordic countries." More specifically in the 
reforms we examined, the informants assured that this was in fact done, 
even if it may not have been translated into bibliometric references. This 
discrepancy can partly be explained by the first narrative, in which coop- 
eration and policy influences are not carefully documented. Another 
explanation for Nordic evidence being tangential at most in the docu- 
ments we scrutinized could be that Nordic cooperation is axiomatic and 
no bibliometric reference is possible or needed. The following two 
Icelandic interview excerpts illustrate this point with lucidity. While 
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discussing updating a law on compulsory education, an Icelandic infor- 
mant noted: 


Thus, it was in all this process that the Nordic laws were always on the 
table, as a sort of a working document, and we used arguments from the 
Nordic background documents. I am pretty certain. But it was such a mat- 
ter of course that it didnt need mentioning. (Civil servant, Iceland) 


In discussing updating the curriculum, an informant stated: 


I am pretty certain, as I remember it, we were all reading the Nordic cur- 
ricula, and this was taken for granted and didn't need discussing. (Civil 
servant, Iceland) 


For some of these Icelandic informants, the Nordic space was such an 
implicit part of the process that it did not need to be acknowledged spe- 
cifically. Furthermore, we encountered similar narratives in the Finnish 
and Norwegian interviews. The narratives we discovered of the Nordic 
space being so implicit yet simultaneously omnipresent bear similarities 
to the description of pre-Lisbon European education space, as described 
by Grek and Rinne (2011). Unlike the European education space, the 
Nordic is not built on harmonization through benchmarking and num- 
bers, but it still pre-exists in the hearts and minds of the people, like the 
pre-Lisbon European space did (Grek & Rinne, 2011, pp. 29-30). As 
such, it is so ubiquitous that no distinct mentioning is required. 
According to several informants, the absence of regional Nordic evi- 
dence may be related to the dynamics of externalization (the inclination 
to refer to the education systems of other countries or information pro- 
duced by global actors in reform proposals, see for instance: Steiner- 
Khamsi, 2003) and to the tendency to maintain the status quo and avoid 
including Nordic knowledge that may prove controversial compared to 
the predominant global policy discourses. In an interview about the 
Norwegian committee work, one informant described the following: 


I think it’s about externalization. I think international references are much 
heavier in those situations. When it comes to writing policy texts, so there 
were a lot of Nordic references in the discussions but yeah, I can also see 
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that in order to get legitimacy for the policymaking international refer- 
ences are playing a much more, a larger role than the Nordic references so, 
and I think it's about not to challenge the consensus too much. If you bring 
in domestic or Nordic references, it could be more controversial because 
the discussions that usually emerge in reform topics are much more a local 
thing. (Committee member, Norway) 


This excerpt illustrates well the third narrative we identified, which is that 
the legitimation function and hierarchization of evidence play roles in 
choosing evidence to reference in written documents. Several informants 
acknowledged that Nordic cooperation does not produce data that can be 
used as references in policy documents, but they also stated that, even if 
it did, international evidence would still enjoy a higher status as a legiti- 
mation instrument. Our informants frequently brought up legitimation 
as an explanation for the absence of Nordic knowledge as evidence. In 
particular, they mentioned the OECD as one of the main providers of 
policy evidence used for legitimation of nationally made decisions. 

These findings are perfectly in line with the theorizations of Wendy 
Espeland (2015) and Deborah Stone (2016), who stated that numbers 
are particularly appealing for policymakers and politicians since their 
ostensible objectivity makes them ideal in legitimating any political argu- 
ment. Ihe OECD produces typically comparative data based on stan- 
dardized indicators detached from both the multiplicities of local cultures 
and situations and the organization's own political agenda. This perceived 
objectivity makes this data an excellent tool for mechanical objectivity 
(Eyal, 2019), as it can be used to match any central topic or issue in the 
national reform processes. 

The OECD also produces an extensive amount of reports and other 
written materials where this data is documented and easily usable for 
bibliometric reference. However, our informants stressed the legitimation 
status and function of the OECD data over the mere technicalities of it 
being carefully documented. In fact, one informant even mentioned that 
country reports were specifically ordered for national legitimation pur- 
poses. The informant questioned the previous research on transnational 
governance, claiming that the data production of the OECD often stems 
from national needs for evidence. The informant further asserted that the 
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OECD S reports are frequently commissioned by national governments 
for specific legitimation needs. He stated: 


Well, that transnational governance, there is of course this point of view 
related to it, that the OECD produces different kinds of data. That there is 
real data that can be analyzed nationally and base the decisions on, to use 
it as basis for decision. But then there are also these country reports that the 
OECD countries can order, so this kind of commissioned research. And it 
is of course so, and we have examples of this as well in our state governance, 
that if not directly that, that they stem from the own interests of the coun- 
try in question. That one decides on a political measure, and then one 
orders the country report from the OECD, where then international evalu- 
ators become involved. And before these reports are published, the coun- 
tries and the OECD do discuss them. So I think this idea of transnational 
governance is just one angle, namely this when one uses these country 
reports. But here one comes to what I said before, the blanks in evaluating 
research, that one legitimizes one’s own views with some material produced 
by an international actor. (Working group member, Finland) 


This interview excerpt illustrates the fourth narrative as well. Most infor- 
mants acknowledged and underlined the fact that in curriculum reforms, 
the national orientation and political dynamics still played the greatest 
role. In general, national core curricula, as they have been described by 
our informants, are securely in the national domain of education policy 
and politics, as this Norwegian informant emphasized: 


Perhaps one of the explanations is that in most countries they probably 
consider such curriculum work as a very national domain. ... Every time 
in a way curricula are put on the agenda internationally it is like—the first 
things everyone says, is that curricula are a national domain. We do not 
want such an international decision on that. That this area is not suitable 
for that. (Committee member, Norway) 


The fact that curriculum development is still seen as a very national 
domain of education policy may well explain not only the absence of 
regional references in our data, but also the self-referential nature of the 
country cases addressed in more detail in the previous chapters. Perhaps 
research on another domain of education policy, for instance higher 
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education policy, would have produced data that included more interna- 
tional and regional references. However, the influence of evidence or the 
best available knowledge did not seem to be the most important factor in 
determining the direction for the reforms. The result of the reforms often 
arose from negotiations, and issues on the reform agenda sometimes 
became politicized. In a conflict or gridlock situation, compromises were 
inevitable, even if it meant not utilizing the best knowledge available, as 
demonstrated in the following interview exchange: 


Interviewer: 


Informant: 


Interviewer: 


Informant: 


Interviewer: 


Informant: 


Do you remember any official Nordic meeting or network that 
you were in contact with during the reform processes? 

No, not directly—not directly in the project [of writing a white 
paper]. But we have working groups that work like that, and 
everything we have written has in a way also been negotiated by 
them. And we also get text input from people who also have this 
as their specialized area. 

I wonder about the process of deciding which sources of knowl- 
edge are relevant to draw on—in a report to the parliament. 
How have you decided which sources you would like to take a 
closer look at and include in your writing process? 

I have actually tried to have as broad an approach as possible to 
see what exists. ... If you look at the assessment chapter, for 
example, I have to see what is the latest and what is really hap- 
pening in that field now. So I have in a way tried to be as broad 
as possible—for the recommendations in the NOU! is one 
thing, and they are evaluated against a knowledge base, too, but 
at the same time we have to see if there are other things, and it is 
often the case that maybe the politicians want something else 
than what is recommended and proposed in the NOU, as well. 

A naive question: is it in the end a process where you look at the 
list and just check if the reference list reveals different selection 
criteria? Or is that list just a result? 

I wish I could answer that [laughs], but I think it's a result. 
(Committee member, Norway) 
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In spite of the narratives explaining the absent Nordic "other" our 
informants described Nordic policy cooperation as a space of oneness. In 
the interviews, many described this space as a feeling of belonging. 
According to our interview findings, the Nordic education space works as 
an arena where issues and policy developments are discussed with the 
like-minded. Still, the actors are simultaneously aware that the education 
systems, the political situations, and even the degree of marketization 
tendencies in the five countries are rather different. Though the coopera- 
tion may not produce actual evidence according to the definition we 
employ in this chapter, the Nordic education space had concrete benefits 
as well according to our informants. These benefits materialized particu- 
larly in the global policy space, for example in the meetings organized by 
the OECD where the Nordic representatives collaborated, for instance, 
by voting for same policy solutions and recommendations. An important 
part of informal cooperation happens in this kind of international set- 
ting, as demonstrated in this excerpt from an Icelandic civil servant: 


Informant: As you look to these countries, then one sees a certain underly- 
ing OECD influence. Some may think that this is a hard line 
coming from above, but that is not correct. These are influences 
formed in unison by the participating countries, and I think this 
has normally been so. I think there are both indirect and direct 
influences from there. You attend twice a year with representa- 
tives from all the countries. There is a special Nordic meeting, 
always a preparatory meeting before the OECD meeting, where 
you discuss with your Nordic colleagues. 

Interviewer: Is it a special meeting, or does it connect to the OECD meeting? 

Informant: There is always a dinner before, and thus you get to know these 
Nordic representatives and people discuss issues. Hardly ever a 
formal Nordic stand is taken but, yes, some coordination. If the 
Nordics want to take the initiative, then it is coordinated there. 

Interviewer: Is this a formal meeting with an agenda? 

Informant: No, this is just an informal dinner. Yes, just discussion. Of 
course, at these [OECD] meetings, there are loads of reports and 
documents, project proposals. The Nordic countries may have 
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decided to take a stand on some emphasis in new projects, as 
there are votes on what to research. 

Interviewer: And this carries with it some influence? Nordic influence? 

Informant: Absolutely. The Nordic countries carry much influence in there. 
They are always, normally, in agreement. One takes the floor, 
and the others support. It carries some weight. Even China or 
large countries like the US, which are always on their own, they 
have their one vote each. Thus, yes, of course, it carries some 
weight. And people notice. (Civil servant, Iceland) 


In this case, the international setting acted as a glue between the Nordic 
countries, since the Nordic representatives met beforehand during an 
informal dinner to discuss the formal issues on the OECD meeting 
agenda. The Nordic community of sameness was, in this setting, a strate- 
gic tool to gain more power in an international setting, as the Nordic 
countries supported each other in official votes. This arrangement indi- 
cates that, regardless of the possible differences in education systems, 
policymaking procedures, and political situations in these countries, they 
share some fundamentally similar values and interests that allow them to 
reach such consensus. The aforementioned excerpt also validates the pre- 
vious claim of several scholars that the Nordic countries are seen as one 
entity from the outside. It demonstrates that other countries and regions 
acknowledge the weight of Nordic unity. 

Moreover, the conduct and practices of the Nordic actors in the inter- 
national policy space also demonstrate how the national, regional, and 
international are not layered and separate. The global dynamics of the 
education space are not just a matter of international organizations exer- 
cising their influence on the national level, with the regional level acting 
possibly as a buffer or a mediator in this process. Instead, our interviews 
painted a picture of an active and rather powerful Nordic education space 
that stretches out through the connections of the national actors and 
exercises its influence as one entity in the international decisions and 
recommendations of the OECD and other organizations. Even if this 
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influence cannot be traced back in the policy documents as bibliometric 
references or as evidence in the way defined by Paul Cairney (2016), the 
Nordic education space has in fact influenced what is included in the 
international policy recommendations (e.g., the OECD reports) now 
used as evidence in the national documents. The regional context may 
shape receptiveness to evidence, but it may also shape the actual sources 
of evidence when they are created and drafted in the international con- 
text of policy cooperation. In short, there certainly is more in the policy 
process than meets the eye. 


Conclusions 


Previous research has implied that neighboring countries may be more 
receptive to the same policy solutions due to their commitment to a 
common culture and similarities in their legal and political systems. The 
Nordic region can be looked upon as a cluster of such countries. They 
are often, both from the inside and the outside, perceived as one cultural 
and political territory. One could thereby expect considerable transfer of 
policy knowledge within this region. However, our findings reveal that 
the Nordic policy cooperation and exchange of policy knowledge in the 
field of education is far more complex and influenced by multitude of 
aspects and dynamics, in addition to cultural, legal, and political 
similarities. 

In this chapter, we have investigated the use of regional Nordic pol- 
icy knowledge in education reforms in three of the Nordic countries: 
Finland, Iceland, and Norway. Our starting point was the definition of 
evidence by Paul Cairney (2016, 3) as “an argument backed by infor- 
mation." By employing both bibliometric network analysis and the- 
matic analysis of expert interviews, we have examined how policymakers 
and experts in these three countries locate themselves in a larger politi- 
cal reference space when they develop ideas and collect evidence to 
justify school reforms in their respective countries. We have asked 
whether government-appointed expert panels and policymakers within 
the state administration consider knowledge and information from 


12 Regional Policy Spaces, Knowledge Networks... 377 


other Nordic countries relevant. Moreover, we examined whether they 
demonstrate a specific interest in authorizing such sources by referenc- 
ing them in public policy, and investigated the factors guiding these 
decisions. 

Our bibliometric network analysis revealed a notable absence of 
Nordic references in the source documents we included in our study. 
Regional Nordic references amounted to only 1-7% of all the references, 
and we found very few references that were cited by more than one coun- 
try. This finding indicates that publications from other Nordic countries 
are not considered relevant in terms of "authorized evidence" that is 
explicitly included in reference lists or footnotes in white and green 
papers. Nevertheless, our interview data shows that Nordic cooperation 
in the field of education policy is vivid, frequent, and ample. Therefore, 
good reasons remain to argue that the expert panels and policymakers 
involved in these reforms are very well informed about emerging trends 
and ongoing reforms in other countries. 

In the interviews, we asked our informants for reasons why Nordic 
knowledge does not appear as the most prominent reference source in 
policy processes; in other words, we wondered why it is not referenced 
and used as evidence in the policy documents. We discovered four main 
explanatory narratives for this phenomenon. First, Nordic cooperation is 
not thoroughly documented nor does it produce data that lends itself 
easily to reference. Second, Nordic cooperation is such an implicit part of 
the policy process that it does not need to be mentioned specifically. The 
third explanatory narrative is related to the legitimation function and 
hierarchization of policy evidence. Even if Nordic cooperation would 
produce data that was easy to reference, international evidence still had a 
higher status as a legitimation device than regional evidence. Finally, the 
fourth explanatory narrative offered by our informants was that national 
orientation and political dynamics still play the greatest role in curricu- 
lum reforms. Despite increasing transnational influences, the curriculum 
is still characterized as a particular domain in education policymaking 
and politics, considered predominantly national by character. Although 
school reforms reflect international influence, the problems addressed 
turn into national concerns when they are handled. In particular, the 
policymakers writing the white papers argue that their job is to bridge the 
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gap between political aims and projects on the one hand and broad exper- 
tise on the other hand, which implies complex processes that involve a 
range of stakeholders. 

In addition to investigating why Nordic evidence is or is not refer- 
enced in the policy documents, our aim in this chapter was to examine 
how policymakers and experts in Finland, Iceland, and Norway locate 
themselves in a larger political reference space. We have developed a 
notion of a Nordic education space in which national experts and policy- 
makers "make meaning and attempt coherence in networked forms" 
(Grek et al., 2020, p. 4). The Nordic policy space is characterized by flu- 
idity. We have shown that there are several forms and networks of Nordic 
cooperation in the field of education policy, and national experts often 
participate in multitudes of networks and forms of cooperation. The ref- 
erence space of the Nordic experts stretches out from the national to the 
transnational through their interactions, connections, and networks. The 
Nordic education policy space is constructed and maintained in different 
settings of cooperation and communication, with some taking place in 
the regional and some in the international domain. When the actors 
meet in international settings, consensus-making is of core importance 
between representatives from the Nordic countries. This differs from 
regional meetings where they rather explore national solutions to com- 
mon problems. Thus, we demonstrate that there are several forms and 
networks of Nordic cooperation that have various functions in terms of 
evidence-based policymaking depending on the context in which col- 
laboration takes place. 

To conclude, the regional Nordic policy space is, in fact, in constant 
movement and a state of becoming. As a result, both comparative research 
and the actors involved in this continuously evolving space are challenged 
to explain its role and function and to pinpoint its changing forms in a 
comprehensive manner. We have aimed to contribute to this discussion, 
but we acknowledge that, in line with the nature of the constant becom- 
ing of this space, we may have posed more questions than we have 
answered. Instead of offering finite conclusions, we have created new 
openings for further discussion. These openings are much like the policy 
space we have examined—in a constant state of becoming and hence 
infinite in nature. 
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Note 


1. NOUs are Norwegian Official Reports. 
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On Evidence, Impact, and Layers 
in Education Policy Processes 


Kerstin Martens 


The editors invited Antoni Verger and me to reflect on this book's contri- 
butions by identifying its connections with existing research, pointing 
out novel ideas, and proposing topics that deserve further scrutiny. I was 
greatly honored to accept this invitation, as it enabled me to participate 
in and contribute to this endeavor on evidence and expertise in the 
Nordic countries. I, therefore, see the purpose of my commentary not as 
summarizing the findings or providing feedback on the chapters con- 
tained within but, rather, as an exercise in examining cross-cutting themes 
and future avenues for research resulting from this book. Hence, my 
comments reflect on the empirics of the chapters in conjunction with the 
guiding theoretical and methodological chapters. A commentary also 
leaves room for one to argue more freely, to point out the societal con- 
texts of an academic work, and to include "unusual" references, as 
opposed to being limited to referencing only academic work. Therefore, 
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some parts of this chapter may seem to stray slightly off the typical aca- 
demic path but do so in the interest of provoking future discussion. 

Overall, from my reflections, three substantial themes emerged that I 
address in this chapter: (1) the conjunction between evidence and poli- 
tics, (2) the conjunction between referencing and impact, and (3) the 
conjunction between empirics and theory in global education research. 
These three linkages seem to be part of a broader connection referenced 
in the book, namely, the connection between science and politics. Science 
has become a point of reference for politicians due to increased awareness 
about the challenges our world faces with respect to climate change; 
moreover, science has become more visible and more clearly defined in 
political processes in recent years. This was particularly apparent during 
the 2020 Democratic National Convention, at which Joe Biden and 
Kamala Harris were formally nominated as the Democratic presidential 
and vice-presidential candidates, respectively. In their remarks at the con- 
vention, many speakers (including, among others, Michelle Obama) fre- 
quently used the term "science" to make political statements, such as 
noting that Joe Biden would listen to “science,” or would follow “sci- 
ence,” or would consider “scientific reasoning,” and so forth. Obviously, 
these comments were meant to sharply contrast with former president 
Donald Trumps "alternative facts." 

However, we must also remember that science is not static; it is subject 
to change when new findings disprove previous knowledge or question 
conventional knowledge, or when new problems occur that old solutions 
cannot resolve.! The coronavirus pandemic is teaching us to question 
ourselves on an almost daily basis. Is the Swedish way of believing in herd 
immunity the better long-term strategy, despite higher casualties in the 
short run? Was the worldwide lockdown sufficient for keeping the num- 
ber of infections under control? Are the economic and psychological con- 
sequences of that lockdown more disastrous for democracies than we 
anticipated? Are locally defined solutions the better way to balance restric- 
tions and daily work life (masks for school children, limitations on how 
many people are allowed to meet, and so on), despite the wide-ranging 
mobility of people commuting long distances and across borders? 

Perhaps we also need to ask ourselves what is special about the social 
sciences. Without going into the philosophy of science, social science 
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knowledge often seems rather cumulative regarding theoretical 
approaches, in that theories that have proven (partly) unhelpful in a par- 
ticular context are not easily discarded. We have not discovered many 
"black swans,” as Popper would claim, but as social scientists we tend to 
acknowledge that different theories may be equally valid, though in dif- 
ferent contexts and at different times. In addition, the social sciences have 
a particular "problem": We deal intensively with humans and human 
behavior. Humans can be highly complex, changing their minds, opin- 
ions, attitudes, and actions at any given time. They also react to their 
environment in a variety of ways. To quote a classic political science jour- 
nal title by Steven Bernstein and colleagues (2000): God Gave Physics the 
Easy Problems. 

As much as we believe in science and research, scientification and aca- 
demization have their limits in the social sciences and in education stud- 
ies, respectively. As my colleague Laura Engel (2016) from George 
Washington University once argued during a workshop in Hanover, 
sabermetrics—understood as a method used in baseball for collecting 
and summarizing all relevant (but also seemingly irrelevant) data, as 
explained in the book Moneyball by Michael Lewis (2003)—cannot eas- 
ily be applied to education. If it were that simple, all we would have to do 
is gather sufficient appropriate data and let statisticians run a couple of 
regressions in order to describe the perfect education system. However, 
that is not the case, so we need to find other and better ways to research 
education policy. This volume can guide future work on global education 
governance due to its empirical findings, its theoretical approach, and its 
methodological finesse. 


Conjunction Between Evidence and Politics 


This book provides a valuable analytic snapshot of knowledge production 
processes, of the scientification of education (policy), and of the path 
through which academic knowledge flows into politics. Referencing and 
analyzing the networks of actors behind these activities provide a deep 
view into legitimization processes in current politics. The Nordic coun- 
tries are characterized as a comparatively homogenous group of states 
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that, despite some differences, take a similar approach to political 
decision-making and have a shared history of evidence-based policymak- 
ing. Therefore, these countries make ideal examples for this investigation 
for clarifying and illuminating significant aspects of referencing. 

From a political science perspective, the dataset of documents on 
which many of the analyses in this book are based not only produces 
valuable results, as demonstrated in the different empirical chapters, but 
also provides a fascinating sample that can be used for additional follow- 
up analyses. All recent school reform acts that are examined in this vol- 
ume were introduced in each of the Nordic countries at roughly, and 
sometimes exactly, the same time: 2014 seems to be the watershed year in 
education politics in this region (with Norway lagging two years behind). 
Thus, from a methodological perspective, the Nordic context comes close 
to what comparativists (in political science) would call an “experimental 
setting," which includes the factors that best support measuring the 
impact of the reforms in each country, as some variables are more or less 
controlled for, including time, institutional background, and regional 
diffusion. Such a setting is not only highly valued methodologically but 
also rarely possible to establish or to find in comparative analyses. 

The existing dataset has provided scholars and researchers with inter- 
esting findings as presented in the different chapters of this volume. 
However, the dataset can also be used to dig deeper into the processes of 
knowledge production and legitimization in political spheres and iden- 
tify cross-cutting themes. The dataset also provides a valuable example of 
how education research can be systematized more rigorously and under 
one common research umbrella that guides multiple analyses. As a next 
step, maintenance of this dataset of documents will preferably be trans- 
formed into a standing project involving the continuous collection of 
documents on school reforms in the respective countries. Such a project 
would enable future generations of scholars to conduct numerous com- 
parative studies or, at some point, studies about individual countries over 
a longer period of time. 

A comparable dataset with which comparative analyses could be envi- 
sioned is, for example, the collection by Marc Helbig and Rita Nikolai 
(20152) on the seemingly “incomparable” German Bundesländer (states) 
with regard to their school politics [Die Unvergleichbaren—Der Wandel 
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der Schulsysteme in den deutschen Bundeslándern seit 1949]. In that work, 
the authors rely on their analysis of around 8000 legal texts from the last 
six decades. The collection of documents is made publicly available for 
future work.’ Thus, this project's approach could serve as a blueprint for 
multiple initiatives: (1) to make the Finnut project (Chap. 1 in this vol- 
ume) public, (2) to simplify the process of finding a possible partner for 
comparative work between the Nordic countries and the German 
Bundeslander, and (3) to develop similar reform datasets for other coun- 
tries or regions of the world. 

However, considering only the dataset on the Nordic states that already 
exists, a vast array of possible new research areas comes to mind. For 
example, the existing databases can be examined for topics and themes 
that regularly appear in the documents. To identify those topics, the data 
can be analyzed quantitatively, for example, using topic modeling, a sta- 
tistical model of machine learning for discovering topics that occur mul- 
tiple times in a collection of documents. In a recent contribution, Helen 
Seitzer et al. (2021) applied topic modeling to examine the OECD'5 
PISA project in relation to its other activities and publications on educa- 
tion to discover if and how things change over time. Surprisingly, they 
find that although the OECD is predominantly renowned for the PISA, 
the themes it deals with are much broader than this, with PISA account- 
ing for only a small part of the OECD’s overall output. In fact, the orga- 
nization addresses higher education policy more intensely than secondary 
school data, even though the OECD and PISA have become almost syn- 
onymous for the international organizations work in education. This 
raises the question as to whether PISA is predominant or perhaps even 
overrated. 

As regards qualitative methodological means, I can also imagine that a 
rather detailed hermeneutic work on the existing dataset, perhaps even in 
cooperation with linguists, would be worthwhile. Single passages, where 
the context in which something is referenced (or not referenced) is evalu- 
ated, could be hermeneutically examined. Something else worth consid- 
ering is the application of a scoring system for statements and references, 
where each could be assessed in its respective context. Furthermore, dig- 
ging deeper into the context of these statements could provide insight 
into connections between themes and produce interesting questions (e.g., 
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does think tank referencing occur more often in the context of equity and 
human capital in relation to the OECD?). Further network analyses 
could also be applied to link topics to references. 


Conjunction Between Referencing and Impact 


With this document dataset, the editors and contributors of this text 
apply a highly sophisticated analysis to the network of referencing in 
Nordic education policy processes. Network analysis is especially suited 
for figurative visualizations of relations, which are often more impressive 
and expressive than those produced by many other analyses. Network 
analysis is also en vogue and innovative in education studies from a meth- 
odological perspective, despite having been established in the fields of 
sociology and political science for some time. Particularly when the net- 
work of actors in global education governance is to be analyzed, a net- 
work approach can make connections or nodes and major actors visible. 
The findings would make interesting contributions to SUNBELT, the 
signature conference of the International Network for Social Network 
Analysis (INSNA). 

However, although network analysis can do more than visualize rela- 
tions between references and point to significant nodes within the net- 
work, centered nodes do not necessarily have the greatest impact. 
Referencing and visualizing relations between references cannot answer 
some of the questions that arise from this research endeavor, such as in 
the context of governmental documents, particularly, the internal impor- 
tance of green and white books, or how much party ideology comes into 
play and how this is pictured or becomes pictured in the references. 
Follow-up questions like those that follow may also emanate from this 
research: Which document(s) were the most influential? Which 
reference(s) have had the most impact? And were these the most cited 
references? 

Possibly even more important for future studies are answers to ques- 
tions about what we do not see in the documents and, accordingly, what 
we do not see in the network analyses: What is the intention behind some 
references being selected and others being (perhaps deliberately) 
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neglected? Considering the somewhat paradigmatic shift brought on by 
PISA toward perceiving educational outcomes as able to be measured 
against globally standardized indicators, the references that were omitted 
and why they were omitted seem to be of even more interest. Where have 
the boundaries of knowledge been set, and what references have been 
hidden? With this in mind, a variety of future studies, both across and 
within countries, is conceivable. 

Nevertheless, one must be aware of the limits of our research endeav- 
ors. Indeed, measuring “impact” is probably one of the most difficult 
tasks we encounter in our work. To specifically pinpoint what factors had 
an impact as well as when, where, and why those impacts were experi- 
enced can usually only be approximated by detailed research. 
Methodologically, and not without advanced reflections on one’s own 
biases, approaches such as process tracing (probably in combination with 
interviews) may prove effective for investigating the steps through which 
as well as the substantial or decisive intention with which these govern- 
ment documents were created. Research questions may include inquiries 
like the following: What were important watershed decisions in the pro- 
cess? How were such decisions made? What were key moments in the 
process? 

Another follow-up step to consider is to examine how these reforms 
were implemented in practice on the ground, for example, in education 
districts, in single schools, and so on, or following the thinking of Gita 
Steiner-Khamsi: How were these reforms translated into the local con- 
text? The written process for implementing the reform does not necessar- 
ily describe the same measures as they were implemented in real life. A 
good example of how differently an international impetus can be trans- 
lated into national and local contexts is "inclusive education." While the 
United Nations (UN) Convention on the Rights of Persons with 
Disabilities counts as the most quickly ratified UN convention, the 
implementation of Article 24 on inclusive education not only varies from 
country to country but—as evident in the example of Germany—also 
from state to state and school to school. Hartong and Nikolai (2017) 
described this as the “local globalness” of policy transfer. 

However, to examine international influences, especially the PISA 
study, the setting applied in this volume provides a sufficient time frame. 
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Around 15 years passed since the first PISA study was published and the 
reform acts of 2014 took place. Thus, considering that political processes 
take time, especially when international influences need to be "trans- 
lated" via national legislation by way of regional administrative bodies 
before even potentially reaching the classroom, the effects of the interna- 
tional impetus may have reached the local realm. All in all, we can assume 
that a PISA-affected generation of students has left school with certifi- 
cates by now. 


Conjunction Between Empirics, Theory, 
and Methods 


Although this book is considered an edited volume, it better resembles a 
monograph in its set-up. As the result of a longer-lasting research project 
(the Finnut Project, Chap. 1), one common theoretical frame is applied, 
including the use of common terminology within the individual chapters 
and an encompassing research design in which each chapter has its dis- 
tinct place. Therefore, the book is not only a rich collection of contribu- 
tions in which each chapter can stand for itself, but the book itself is more 
than the sum of its parts. 

This highly sophisticated approach becomes particularly apparent in 
the systematic and—what I would call—/ayered approach. While Part 1 
of the book provides an overview of the study, key concepts, and the 
methodology applied in the book, Parts 2 and 3 address the empirical 
findings for different country and cross-country comparisons. In doing 
so, different policy layers are explored—the international or transnational 
layer (particularly the OECD in this context as the policy source), the 
national layer (where governments as executors of policies are involved), 
and the layer of societal actors of different kinds (such as trade unions, 
think tanks, and the like)—all of which are part of the discursive policy 
practice. 

This kind of approach within a defined region of the world, such as the 
Nordic countries, enables us to detect not only patterns and outliers 
across countries (horizontal approach) but also configurations and 
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alignments across layers (vertical approach). In my view, this is a highly 
systematic approach that can serve as a methodological tool or as an ele- 
ment of a research design in many other contexts and for other compari- 
sons. Such an approach could also be extended to analyze the activities of 
additional global actors, for example, international organizations, such as 
the World Bank or the UN Educational, Scientific and Cultural 
Organization (UNESCO), or more regional organizations active in edu- 
cation, such as the Southeast Asian Ministers of Education Organization 
(SEAMEO) for the Southeast Asian region or the Arab League 
Educational, Cultural and Scientific Organization (ALECSO) for the 
Arab and Middle East region (see Niemann & Martens, 2021). 

Moreover, an additional layer should be considered and can be added 
to future analyses, namely, individuals. Within the chapters, individuals 
and their particular impact are sporadically mentioned (particularly in 
Chap. 10, and even more so in Chap. 11); sometimes a commission is 
named after them, and sometimes their influence is hinted at within a 
particular context. In general, I believe that we underestimate how influ- 
ential some individuals may be when we discuss the "Ministry of 
Education,” the “lobby group,” or the “education commission." We may 
only know anecdotally that a particular person shaped the context of a 
policy document or the direction of a policy. Various studies in the con- 
text of the OECD and education policy, however, have shown just how 
influential certain individuals have been at key moments (e.g., in Henry 
et al. 2001; Martens, 2007). As an example, one can imagine how PISA 
would look or be perceived without its front man, Andreas Schleicher. 
Thus, more systematic research on the influential role of individuals in 
policy processes should be conducted. 

In addition to this supplementary layer to be applied to future schol- 
arly work in the field of education policy, another source of inspiration 
may be found in the deeper systematic linking of these layers with one 
another and with theory-driven approaches. What the editors produced 
with this volume is what comparativists, especially from political science, 
call a Most Similar Systems Design (MSSD). Comparing a group of simi- 
lar countries, here the Nordic countries, in a small-N analysis allows for 
factors that are historically or institutionally similar in the defined sample 
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to be excluded from the analysis in order to concentrate on differences 
between cases. 

Concentrating on such differences between otherwise similar systems 
also allows for contextualizing them in existing approaches that have 
proven suitable in other studies. Moreover, connecting a MSSD system- 
atically with theoretical approaches that emphasize differences can allow 
us to trace mechanisms and detect causations. They may also deliver the- 
oretically guided hypotheses, which incorporate the different layers and 
their impact on education policy processes. Depending on the particular 
research question, a variety of approaches can provide insightful inter- 
linkages and explanations for phenomena of interest, such as the follow- 
ing: approaches to delta-convergence (for linking the international and 
the national level); welfare state theories and education (for linking the 
type of system to policy outputs and outcomes); and interest group 
involvement, such as in the form of corporations or social movements 
(for linking the national level to internal actors). 


Conclusion 


This volume provides, without question, a particularly rich collection of 
evidence and expertise on Nordic education policy processes. It includes 
representation by a pronounced community of scholars from the region 
who contributed their knowledge to this common endeavor. With this 
volume, the editors and contributors deliver a systematic and well-written 
work of research that stands for itself but will also inspire future studies. 
My commentary focused on three conjunctions that I detected within 
and across the chapters: the conjunction between evidence and politics, 
between referencing and impact, and between theory and empirics. By 
summarizing some of the approaches and findings of the work in this 
way, my aim was also to identify and inspire possible projects for 
the future. 
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Notes 


1. Making such a statement seems to be a delicate matter, as recently dis- 
cussed in the German context. In a podcast for the German Science 
Foundation’s (Deutsche Forschungsgemeinschaft, DFG) online campaign 
#fiirdas Wissen (#forknowledge) on the 100th anniversary of the founda- 
tion’s predecessor organization, German satirist and cabaret artist Dieter 
Nuhr (2020) stated: “For science is no doctrine of salvation, no religion 
that proclaims absolute truths” [Wissenschaft ist nämlich keine Heilslehre, 
keine Religion, die absolute Wahrheiten verkündet]. A real firestorm fol- 
lowed on Twitter, which consequently led the DFG to first withdraw the 
contribution but then later to apologize and post it again. https://dfg2020. 
de/gemeinsam-fuer-das-wissen/ 

2. https://www.pedocs.de/frontdoor.php?source_opus=11096 (Helbig and 
Nikolai 2015b) 
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Evidence-Based Policy Making 
and Educational Reform in Nordic 
Europe: Key Contributions 
of the POLNET Study 


Antoni Verger 


Evidence-based policy making (EBPM) is currently considered the most 
appropriate approach to public policy formulation. EBPM implies that 
both policy decisions and policy changes are increasingly grounded on 
scientific reasoning and research findings. The most enthusiastic advo- 
cates of the evidence shift in policy applaud the fact that policy decisions 
are increasingly informed by science in contrast to anecdotal informa- 
tion, tacit knowledge, public sentiment, and other forms of support. 
EBPM is also considered suitable for addressing ideological biases in 
decision making and for generating broader political consensus on the 
strategies that should guide policy formulation in different domains. In 
the 1990s, both policy and academic circles in the field of education 
began to embrace EBPM to address a legitimacy crisis faced by educa- 
tional research (Head, 2008). To mitigate this crisis, educational research 
borrowed rationales and methods from research in medicine and other 
experimental sciences, with the expectation of promoting a more 
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cumulative and transferable type of knowledge. Scholars and universities 
around the world have welcomed the EBPM transition because it makes 
academic research more socially valuable and visible and, importantly, 
because it generates new venues for research funding. 

To a great extent, evidence-based policy creates opportunities for high 
quality, knowledgeable policy debates—that is to say, policy debates 
informed by causal beliefs, sound policy evaluations, and theoretically 
informed case studies. Politically speaking, EBPM is expected to be 
embraced in an international environment characterized by the resur- 
gence of populist movements for which mistrust in science pays off polit- 
ically and in an environment in which policy decisions on sensitive issues, 
such as global pandemics or climate change, can put population and the 
future of the planet at risk when those decisions are based on "alternative 
facts" (see Marten's commentary chapter in this volume). 

Nevertheless, despite the many ways that paying more attention to 
scientific evidence benefits policy making, EBPM still must be approached 
with caution, as it has its own limitations and risks for both policy 
research and practice. Furthermore, some assumptions behind enthusias- 
tic versions of EBPM may not work as expected in real educational policy 
settings. EBPM discourses assume that, with the emergence of this policy 
approach, political values, ideologies, and normative beliefs are being 
sidelined from policy processes in favor of scientific knowledge and causal 
beliefs. They also assume that EBPM can make policy processes more 
open to external participation, especially to academic voices and sources. 
In this respect, EBPM implies that the locus of decision-making shifts 
from political authorities to networks of experts. Additionally, EBPM 
embraces a rationalistic ontology to policy transfer according to which 
policy learning is the main mechanism behind the traveling and selection 
of “best practices” in different policy contexts. This implies, to a degree, 
acknowledging that policy learning has gained centrality as a mechanism 
of policy transfer against other well-known transfer mechanisms (such as 
competition, coercion, or emulation). 

The book you have in your hands problematizes these and other 
assumptions about EBPM through comparative research conducted in 
the Nordic education policy space. The book is part of a five-year research 
program called POLNET (Policy Knowledge and Lesson Drawing in Nordic 
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School Reform in an Era of International Comparison) investigating the 
implementation of the EBPM approach in contemporary educational 
reforms in Nordic European countries. Ihe POLNET study has brought 
together an international team of outstanding researchers from Nordic 
European universities and the Teachers College (University of Columbia), 
under the leadership of Kirsten Sivesind, Berit Karseth, and Gita Steiner- 
Khamsi. One of the main originalities of this trans-Atlantic research 
effort is the application of a bibliometric methodology to conduct com- 
parative analyses of educational country reforms. That the chapters are 
grounded on this same methodology contributes to making the volume a 
cohesive piece of research and strengthens the comparability of the coun- 
try cases. In the following sections, I reflect on the most important con- 
tributions of the study and their implications for future research. 


Science and Politics in Educational Reform: 
Does Science Rule? 


One of the most persistent critiques of EBPM is that it runs the risk of 
expertizing policy processes and of generating new forms of social closure 
within policy networks. According to Biesta (2007, p. 1), through the 
technification of certain policy debates, EBPM “restricts the opportuni- 
ties for participation in educational decision making.” The POLNET 
study shows that this would not be the case in most Nordic countries, 
where EBPM challenges a corporatist form of governance and promotes 
the transition toward forms of network governance in which new actors— 
such as experts, interest groups, and civil society organizations—are con- 
sidered in policy formulation processes. In fact, since Nordic governments 
have embraced EBPM in educational reform, parliamentary debates on 
education policy have intensified. Thus, the policy process is becoming 
more transparent, open, and political, while being configured through 
the involvement of a wider nebulous of actors, spaces, and devices for 
participation (including conferences, commissions, social media interac- 
tions, position papers, public blogs, and so on). 

Pro-EBPM discourses celebrate the increasing influence of science over 
politics in policy making and the scientization of policy debates that were 


398 A. Verger 


considered too political. However, the POLNET study demonstrates 
that, in policy processes, the relationship between science and politics is 
not necessarily a zero-sum game. The educational reforms analyzed in the 
book, in fact, show that the bigger presence of science has, indeed, evolved 
in parallel to more political participation and to the politicization of 
some stages of the reform processes. 

Certainly, in real-life situations the separation between politics and 
science is precarious and far from linear. Chapter 10 in this text develops 
this idea eloquently; it graphically depicts the recursive and constant iter- 
ation between the political and scientific domains in education policy 
making (see Fig. 10.8). In the words of its authors, "information- 
gathering and consensus-building occur in practice at each and every step 
of the policy making process, blurring the line between science and 
politics." 

‘These findings confirm that scientific evidence is not ontologically dis- 
tinguishable from political ideologies, normative beliefs, and bureaucratic 
control in policy-making processes. As illustrated by the POLNET study, 
the institutionalized forms of EBPM in different Nordic countries make 
clear that politics are at play in decisions that alter the influence of science 
over policy at many levels. To start with, by designing, regulating, and 
funding the EBPM architecture, political actors can dynamically condi- 
tion the policy outcomes of evidence-based deliberation processes. 
Decisions regarding the configuration and funding of the agencies in 
charge of drafting green papers, literature reviews, and position papers are 
key to understanding the dialectal relationship between science and poli- 
tics in policy processes, as are decisions about who can integrate the agen- 
cies or advisory commissions (including the identification of criteria 
regarding disciplinary background or research experience). As principal- 
agent models would predict, these decisions have important implications 
for the outcomes of EBPM schemes, as well as for the actual role of inde- 
pendent research in educational reform. 

However, politics also condition and, to an extent, restrict scientific 
influence over policy in more indirect ways. EBPM assumes that policy 
makers, especially in periods of uncertainty and crisis, are receptive to 
scientific evidence on best practices. Thus, policy makers increasingly 
welcome the role of knowledge brokers and experts when selecting new 


14 Evidence-Based Policy Making and Educational Reform... 399 


policy instruments or calibrating those that already exist. Yet, they are 
also aware of the type of knowledge that is more useful to them in both 
political and policy terms and have at least notions of who can provide 
them with this kind of knowledge more effectively. Specifically, policy 
makers tend to resort to research sources where they can obtain straight- 
forward answers to frequently complex policy problems—sources that fit 
within what Roger Dale (1994) defined as a “problem-solving” type of 
research. Policy makers usually rely on technical knowledge to address 
“what works” questions and are more inclined to base their assessments 
on quantitative sources rather than on qualitative research and data, with 
the latter often seen as more interpretative and biased. In doing so, they 
reproduce ideational frameworks of what type of knowledge counts as 
policy-actionable evidence. Discursive selectivity, hence, becomes a sub- 
tle mechanism for understanding how politics shapes EBPM processes 
and privileges certain research approaches over others. 

Not surprisingly, many of the references cited in the green papers pub- 
lished in the Nordic countries analyzed fit within a mainstream “school 
effectiveness research” approach—an approach that pays more attention 
to measure the school effects on student learning than to multiple causal 
and structural explanations of educational outcomes (Parra, 2018). 
International scholars with a pragmatic approach to educational research 
who prescribe clear policy guidelines on how to organize effective schools 
and classrooms, such as John Hatti and Michael Fullan, are more often 
cited in the policy papers reviewed in the POLNET study than scholars 
with a more critical or theoretical understanding of education policy 
matters. 

Nonetheless, a finding as unexpected as it is important from the 
POLNET study is that references to international and national educa- 
tional researchers are relatively scarce in the policy documents used to 
back educational reform. Even though the green and white papers pub- 
lished in the Nordic countries analyzed cite numerous pieces of evidence, 
academic evidence in the form of publications in peer-reviewed journals 
is rather marginal. In contrast, other types of domestic publications (gray 
literature, ad hoc literature reviews, blog posts, and so on) are much more 
present. The response to the question “whose knowledge is used in edu- 
cational reform in Nordic countries?” is not straightforward; indeed, it 
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seems easier to respond to the question of whose knowledge is not used. 
Educational research produced in national universities is not strongly 
represented in most of the country cases. 


Externalization and the Role 
of International Organizations 


In contrast to the marginal presence of academic publications produced 
by national universities, reports and papers published by the Organization 
for Economic Co-operation and Development (OECD) are widely cited 
in the policy documents produced by state agencies and research insti- 
tutes in all Nordic countries. The OECD is not only the most cited inter- 
national source in the corpus of policy reports gathered: the bibliometric 
analysis conducted by the POLNET team shows that OECD reports 
enjoy a high level of in-degree centrality in the networks of publications 
produced in relation to the different country contexts. This confirms the 
OECD’s authority in the education policy realm, and in Nordic European 
education in particular (Grek, 2017; Ydesen, 2019). Still, as acknowl- 
edged in Chap. 11, the bibliometric analysis helps to test the reputation 
of the OECD in the Nordic region but does not capture, on its own, the 
level of penetration of the OECD in the Nordic education policy space 
neither the nature of this international organizations influence in national 
policies. 

OECD reports can be cited in green or white papers for multiple rea- 
sons, not all of which are related to the ideational influence of this inter- 
national organization—or to how much national policy actors have 
learned from or within OECD initiatives. In the context of educational 
reforms, governments may cite OECD sources to legitimize their policy 
options, and impregnate these options with international status. 
Moreover, as the authors of Chap. 6 wittily observe, referring to external 
sources such as the OECD in a country like Iceland, where policy net- 
works are rather compact, is simply a way to "avoid being accused of 


14 Evidence-Based Policy Making and Educational Reform... 401 


nepotism." Countries can also cite OECD papers to technify political 
debates considered too controversial in the domestic policy arena (Rizvi 
& Lingard, 2009; Browes & Verger, 2020). In fact, as the authors of 
Chap. 12 observe, the number of references to the OECD increases in 
those countries in which education reforms have been more contentious. 
To a degree, externalization intensifies when more sensitive or controver- 
sial reforms are being debated in national policy spaces. All of this cor- 
roborates what Martin Marcussen observed some time ago: the fact that 
countries increasingly resort to and cite OECD knowledge products says 
more about the increasing international legitimacy of this international 
organization than about its policy influence (Marcussen, 2004). 

The OECD has a longstanding legitimacy in Nordic countries. This 
international organization has been present in education policy delibera- 
tions in the region since the 1990s; it has conducted education policy 
reviews in several countries repeatedly; and has involved them in different 
training and research initiatives. However, knowledge dissemination and 
policy evaluations are not the only, or the most important, ways in which 
the OECD has affected national policy debate in Nordic Europe. As sev- 
eral chapters in this volume reflect, the OECD has influenced substantive 
policy change through the “scandalization” that came with the Programme 
for International Student Assessment (PISA) (cf. Steiner-Khamsi & 
Waldow, 2018). Specifically, because of the PISA effect, many Nordic 
countries have adopted performance-based accountability, standardized 
testing instruments, and curricular standards that were not in place before 
(see, for instance, Camphuijsen et al., 2020). International performance 
data has had a significant affective type of impact as a catalyst of educa- 
tional reform (see Sellar & Lingard, 2018). In other words, the adoption 
of the reforms as described resulted more from governments’ political 
and, to some extent, “emotional” reactions to poor (or, rather, lower than 
expected) international performance on PISA than to scientific-based 
deliberations about policy. 

These policy dynamics do not neglect that EBPM may play a role in 
decisions on how to calibrate “governance by numbers” instruments. 
What they highlight is that the substantive decision on whether to adopt 
the “governance by numbers” approach and related policy instruments in 
Nordic countries is more related to the between-countries competition 


402 A. Verger 


and to the anxieties over educational performance that international 
large-scale assessments such as PISA have triggered. 


Is There a Nordic Model of EBPM? 


The POLNET study also examines education policy transfer within the 
Nordic region. The study findings do not support that policy transfer 
dynamics within the region have shaped country reforms. Specifically, 
the number of references to other countries sources in educational reform 
processes is quite low in the documents analyzed: between 196—796 of all 
references (see Chap. 12 in this volume). Even references to Finland, a 
reference educational system at the global level, are very low in the Nordic 
region. Nevertheless, again, these results need to be considered with cau- 
tion because policy transfer mechanisms may operate via more subtle 
forms and informal relations than through explicit mention in policy 
documents. 

At the same time, this book has much to say about the Nordic model 
of education as such, as well as about the predominant model of EBPM 
that is being enacted within the region. For external observers, the image 
of educational systems in Nordic countries includes policy principles 
such as equity, comprehensiveness, decentralization, publicness, and 
teachers professional autonomy. However, as several chapters in this 
book highlight, the Nordic education model is no longer as cohesive. 
Growing marketization, school choice demands, and performance-based 
accountability are transforming the Nordic educational model (with 
Sweden as the regional outlier in advancing these trends). The reforms 
analyzed in this book provide good examples of the tensions that co-exist 
within the Nordic model. These reforms tense educational systems 
by including policy measures that, on one hand, strengthen the knowl- 
edge base of teachers but that, on the other hand, challenge teachers as 
the main source of educational expertise. They also include measures that 
promote equity and inclusion and, at the same time, strengthen competi- 
tive and performative attitudes within schools. Overall, contemporary 
reforms in Nordic countries have stressed educational systems by 
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reinforcing a post-bureaucratic governance approach that is re-scaling 
power in education upwards (from teachers to local and national 
authorities). 

Nordic countries are also known internationally for their well- 
functioning democracies and respect for political rights. Despite the limi- 
tations of EBPM highlighted in the book, the education policy-making 
processes described reflect and attempt to advance more transparent edu- 
cational reforms that are open to external voices. As an educational policy 
researcher from a different world region with a more immature democ- 
racy, I find remarkable how transparent and, to some extent, Cartesian 
policy formulation processes in the Nordic region seem to be, and the 
accessibility of the sources of knowledge on which reforms are to be 
grounded. Citations may be selective and used for legitimation purposes. 
State agencies and research institutes, in their role as knowledge brokers, 
can act as gatekeepers and exclude certain voices. But explicit effort to 
back policy change with evidence is evident. Thus, the opportunity exists 
to tell policy makers whether they have misinterpreted research findings 
or that they have obviated important publications on specific topics. 
In contrast, in contexts where policy making seems to more closely follow 
the garbage-can model than the EBPM model, the opportunities to 
mobilize knowledge for policy change purposes are much more restricted. 

The POLNET study finds some regional citation patterns in the 
reports produced by state agencies and research institutes in preparation 
for educational reform. 'Ihe first is the above-mentioned marginal pres- 
ence of academic research produced by national universities; the second 
is the predominant presence of national sources (in contrast to regional 
sources); and the third is the important presence of OECD reports and 
other knowledge products from this international organization. The con- 
tributors to the book also identified differences regarding the EBPM 
approach in the Nordic countries. Bibliometric analysis is a useful tool to 
make sense of the particularities of the EBPM process that each country 
has followed and identify variation regarding the number of written 
sources used in policy papers, the concrete percentage of academic and 
international sources cited, and the level of externalization on interna- 
tional organizations. 
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The case of Denmark is interesting in this regard. The POLNET study 
shows that education reform in this country has been backed by a limited 
number of written sources but has been heavily influenced by the OECD 
(see Chap. 4 in this volume). However, what makes this case somewhat 
exceptional in the Nordic context is that not only has EBPM contributed 
to the internationalization of the policy process, but the EBPM infra- 
structure itself has become highly influenced by international sources, 
specifically, through the direct involvement of UK players, such as the 
Evidence for Policy and Practice Information and Co-ordinating Centre 
(EPPI) of the University College London-Institute of Education. 


To Conclude: Future Research Directions 


Advocates of EBPM assume that this policy approach favors more open 
and participatory policy processes in which deliberation based on aca- 
demic sources intensifies. They also assume policy learning is the main 
mechanism of policy transfer. In this chapter, I explained how the 
POLNET study challenges these and other assumptions through original 
comparative research conducted in the Nordic education policy space. 
The study shows that the enactment of EBPM has not flattened the ter- 
rain of policy making, nor is it a synonym for the increasing influence of 
science over politics. In the Nordic region, EBPM translates into a com- 
plex policy process in which science and politics interact at different lev- 
els but in such a way that the political domain retains bureaucratic control 
over the policy process and its main outcomes. The study also demon- 
strates that, more than policy learning, dynamics of legitimation and 
competition are key to understanding some of the most substantive pol- 
icy changes Nordic countries have undergone in the educational realm in 
the last two decades. 

The POLNET study addresses numerous research questions, but as 
happens with good research, it also generates new questions and ideas 
about future lines of inquiry. In numerous chapters of the book, the 
authors show that scientific knowledge (in particular, the education 
research produced in national universities) is not a main source of politi- 
cal authority in national education reforms. Overall, the research 
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products cited in policy documents seem to play a bigger role in legiti- 
mizing policy decisions than in shaping them. However, under what con- 
ditions would policy makers be more genuinely receptive to scientific 
evidence? Can changes in the public regulation of state agencies and 
research institutes contribute to promote more pluralistic approaches to 
evidence use? Nonetheless, to be fair, the challenges of EBPM implemen- 
tation do not only originate in the field of politics. Indeed, the very 
knowledge base of educational policy research challenges the comprehen- 
sive use of research for policy making. Ihe many themes on educational 
policy in which research evidence is inconclusive (such as the costs and 
benefits of pedagogic innovation, school choice, performance-based 
accountability, public-private partnerships, etc.) is conducive to both 
knowledge selection biases and the instrumentalization of research for 
political purposes. To test this statement, future research could analyze 
whether EBPM is more genuine and rigorous in relation to those reform 
domains with more cumulative and conclusive research results. 

Another question to unpack EBPM processes is: at what stage of the 
policy process can research evidence become more influential? As we have 
seen, in mainstream EBPM frameworks, the soundest and most welcome 
scientific evidence comes from “problem-solving” research that mainly 
informs policy formulation at the policy design stage; in particular, this is 
research that focuses on school-level effects over learning and packages 
and sells policy solutions in a rather prescriptive way. However, other 
types of research can also play a role in policy processes at different stages, 
although this role tends to be less acknowledged. Numerous research ini- 
tiatives in education are better equipped to problematize existing situa- 
tions in the educational realm than to prescribe straight-forward solutions. 
This, for instance, is the case of research that focuses on identifying the 
problems that policy makers need to address, or on constructing policy 
priorities and preferences that are not central in public agendas yet. 
Future studies on knowledge uses in policy processes could, thus, pay 
more attention to the role of research evidence, not only at the policy 
formulation stage, but also in terms of agenda-setting and 
problematization. 

The fact that the boundaries between the sites of knowledge produc- 
tion and policy making are being blurred, as the POLNET study reveals, 
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means that the corporatist state may not have been totally left behind in 
Nordic countries. To some extent, more than entirely over, the corpora- 
tive tradition has mutated, and the interest groups to which the state 
resorts to in policy processes have been transformed or, in some cases, 
replaced. Something I find intriguing in this regard is the role of teachers 
unions and the political representation of teachers more broadly speak- 
ing. Have teachers’ unions and other teachers’ representatives lost politi- 
cal centrality in the current scenario? Have they been sidelined by the 
growing centrality of research institutes, state agencies, or new interest 
groups? More research on the changing role of teachers’ unions within 
EBPM frameworks, and on how and whether EBPM has transformed the 
functions, power, and nature of collective action within unions, would be 
welcome as well. 

Overall, publications, citations, and references constitute a good entry 
point for the study of the knowledge base of educational reform. 
Nonetheless, as the POLNET study makes clear, publications and refer- 
ences are only the tip of the iceberg—the most visible and empirically 
tangible resource—of more profound and determining political and 
knowledge mobilization dynamics over policy processes. The interest in 
bibliometric analysis is obvious; however, combining it with other meth- 
ods can boost the potential of this methodology. The most informative 
chapters in the book are, in fact, those that combine bibliometric data 
with interview data or that situate bibliometric findings within broader 
research frameworks. Future research can combine different forms of 
social network analysis (including those drawing on bibliometric analy- 
sis) with a more qualitative understanding of the EBPM phenomenon. 
In-depth interviews or observational methods along the lines of political 
ethnography approaches (see Papanastasiou, 2020) can provide insights 
into how policy makers make sense of scientific evidence, the type of 
evidence they find more approachable, and the forms of knowledge to 
which they resort to construct notions of best practice in public policy. 
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Conclusion: Toward a Renewed 
Understanding of Evidence-Based Policy 
in Education 


Berit Karseth and Kirsten Sivesind 


Evidence-Based Policy Advice and Decisions 
in the Nordic Region 


In reviewing the research literature on evidence-based policy, very little 
was found about reference use in policy documentation (see Chaps. 1 
and 2). Therefore, it is interesting to conduct research on how various 
knowledge sources are mobilized in reference networks through policy- 
making processes. Such sources, as mentioned in the introductory chap- 
ter, can be widely defined, including information, ideas, and arguments; 
well-tested beliefs; and lay, professional, and academic knowledge 
(Radaelli, 1995). 

The POLNET study focuses on the use of knowledge sources refer- 
enced within white and green papers. In our study, we found that, in all 
five countries, documents made explicit references to a variety of knowl- 
edge sources. The results of our investigation show that the policy papers 
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reference both governmental documents and draw on other types of doc- 
umentation published by non-academic authors alongside research-based 
evidence produced by various research institutions. However, a striking 
similarity between the five cases is the near absence of academic refer- 
ences from the educational sciences. 

As formulated in the Icelandic chapter, academic papers "are thought 
to be irrelevant or not providing 'accessible knowledge in the evidence 
base for the policy" (Chap. 6, p. 174). Likewise, as spelled out in the 
Danish chapter, references to Danish academic research are almost non- 
existent. Due to international policies that recommend using big data 
and empirical evidence and the pattern of the relatively few references to 
academic works in the educational sciences, there seems to be a mismatch 
between what policy makers and experts consider relevant to cite and 
what, for example, students at universities and colleges read and discuss 
to achieve their degrees in the educational sciences. There are also rela- 
tively few references to articles printed by renowned, national, and inter- 
national publishers. 

Educational science in Europe and the Nordic countries in particular 
has, for institutional reasons, been characterized by professional-practical 
scholarship relevant in, for example, teacher education (Heggen et al., 
2010). Educational scholarship has also evolved as an academic research 
field at universities since the early 1900. Yet, traditions for large-scale 
empirical research vary between areas of expertise in education as well as 
between countries. There is, for example, a long-standing tradition for 
comparative research studies in Sweden and Finland, while Denmark, 
Iceland, and Norway are latecomers in this respect (Sivesind, 2019). 

A surprising observation based on our dataset is the outstanding dis- 
tinction between the numbers of references cited in each of the cases. 
Norway is clearly on top (2312 references), followed by Sweden (1421), 
and Finland (677). The Danish and Icelandic cases have 231 and 203 
references, respectively. One obvious explanation for this difference is the 
selection procedures for collecting documents. In Norway, the team 
decided to start with two white papers and the green papers on basic 
education referenced in the white papers and thereafter count all refer- 
ences within this corpus of documents. The same selection procedure was 
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applied in all countries, but the reforms under study did not produce the 
same number of source documents (i.e., white papers). 

In the particular reform process in Norway, the parliament asked first 
for one white paper on reforming the national curricula and second for a 
white paper that included recommendations for renewing the assessment 
system. The minister in education decided to meet this last request by 
including reform-related themes in a white paper in progress. This 
national case resulted in a larger number of references compared to the 
other four cases. Due to these concrete circumstances, we cannot con- 
clude that the large variations in the number of references represent an 
institutionalized pattern. That is, we have studied single reforms in five 
Nordic countries, and there are good reasons to think of future reforms 
that will result in other patterns and numbers. Nonetheless, there are 
obviously various national traditions and institutional practices of policy 
making within the five countries that result in a limited number of refer- 
ences, such as in Iceland, and many references, documented in the refer- 
ence lists and footnotes, such as in Norway. As Christensen and 
Holst (2017) and Christensen and Hesstvedt (2019) have concluded 
based on their longitudinal survey of public enquiry reports, the experti- 
zation of public enquiry bodies represents an emerging trend in Norway 
that results in an increasing number of references. 

Moreover, as clearly expressed in the Danish chapter, the low number 
of references in this case does not necessarily reflect that the reform did 
not rely on any evidence. Based on additional qualitative data, the authors 
argue that stakeholder evidence and practice-based evidence have been of 
core significance in the preparation of the Danish reform. However, this 
evidence was not substantiated in formal documentation and thereby not 
available to map in quantitative terms with a bibliometric study. So far, 
our observation of how Danish governments authorize their national 
school reforms, seems to be a consequence of a lack of national—institu- 
tionalized procedures to write public enquiry reports, like those in 
Norway or Sweden. 

Nonetheless, the relative number of references to domestic govern- 
mental documents compared to other types of references is significant in 
all five countries. However, there are also vast distinctions for this dimen- 
sion (from around 25% to almost 6%). Sweden and Iceland are on top, 
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Denmark somewhere in the middle, and Norway and Finland at the bot- 
tom. As indicated in more detail in each of the chapters, these disparities 
must be understood and interpreted in the context of national reforms 
that in all countries are launched under the auspices of national govern- 
ments. Nevertheless, independent of the institutional arrangement of the 
bodies that provide the reports, both policy makers and experts are for- 
mally expected to reference government-published documents in all 
Nordic countries. 

Our bibliometric analysis reveals the active utilization of international 
references (cf. Chap. 9). Also for this dimension, our analysis unravels 
interesting differences. While Denmark is the country case with the most 
international references (36.3696), Sweden represents the case with the 
least amount (18.9396). As Nordin and Wahlstróm conclude in their 
chapter, “[T]his finding shows the possibility for national politics to 
uphold a high level of self-referentiality even when the national political 
agenda to a large extent is dictated by international organizations such as 
the OECD” (p. 244). In the Finnish chapter, the authors write that they 
expected to see the significant use of international sources for policy evi- 
dence and, in particular, the use of the OECD; however, data from 
Finland indicates a strong state involvement and concentration of exper- 
tise in state-funded bodies rather than a trust in international expertise. 
Interestingly, in Finland, there is a longstanding tradition of empirical 
research in the field of education and therefore, perhaps, not the same 
need to draw on OECD studies such as in Iceland, Denmark, and 
Norway. Another finding that created a puzzle for the research team was 
the (mostly) absent use of regional Nordic references in the source docu- 
ments (cf. Chap. 12). Although Norway stands out as an exception 
among these countries, with its regional references amounting to almost 
796, this is lower than the authors expected based on previous research on 
the Nordic education model, which describes the region as a com- 
mon unit. 

Taken together, the bibliometric analyses provide important insights 
into how policy makers use various types of evidence to inform, back up, 
and legitimate school reforms. While some findings are in line with what 
other researchers have pointed to, that there is a commonness in terms of 
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how the Nordic countries organize their education system (Telhaug et al., 
2006), there are also results in our study that nuance and even contradict 
the idea of a Nordic education model and, thereby, ideas of unity and 
commonness. Although Nordic policy makers and experts meet in an 
international context to reach a consensus as stakeholders for advocating 
their views, they also act highly self-referentially in regional and national 
settings to deliberate on various possible solutions to their own national 
problems (see Volmari et al., Chap. 12). Therefore, we consider the 
national and comparative chapters in this book to provide complemen- 
tary insights. Reforms are legitimized by national knowledge, which in 
the Nordic countries are authorized by the state, while global and inter- 
national knowledge providers place fingerprints on the reforms through 
their soft governance systems. 


The Constellation of Knowledge Providers 
Within a Nordic Policymaking Context 


As Steiner-Khamsi indicates in Chap. 2, references in terms of citations 
carry epistemological connotations that represent various forms of knowl- 
edge as well as sites for knowledge production that connect national, 
regional, and global policies. The constellation of knowledge providers in 
policymaking processes is thereby changing because of new ideas, institu- 
tions, and networks that characterize both national and transnational 
policy (Legrand, 2021). Due to new partnerships and networks that 
change customary procedures for policy development, the relevance of 
the traditional distinction between applied research, produced by multi- 
ple research institute types, and basic research, produced by universities, 
is challenged (Gibbons et al., 1994; Nowotny et al., 2003; Stokes, 1997). 

In some Nordic countries, governments are funding research programs 
that involve both universities and research institutes in the same machin- 
ery of producing research, expected to make an impact on policymaking 
processes. A core aspiration of these programs, auspices by, for example, 
research councils, is to provide policy-relevant evidence. These programs 
may well transcend traditional boundaries between basic research and 
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applied research, since evaluations assess the researchers and their research 
impact according to the same standards (Smith et al., 2020). 

In tandem with new types of research and evaluation programs, the 
configuration of actors participating in a country's policy advisory system 
is also in transition (Christensen and Holst 2017; Steiner-Khamsi et al. 
2020). As demonstrated in both the national chapters and comparative 
chapters, the national governments and their state agencies are them- 
selves significant knowledge providers. According to Baek et al. (see 
Chap. 9), government papers that summarize information as well as 
research comprise more than one quarter of the references in the Nordic 
sample. In the Swedish case, Nordin and Wahlstróm show that the state 
has produced more than half of the references, representing a system of 
self-referentiality (see Chap. 8). The role of references produced by gov- 
ernments and their policy departments and national agencies underscores 
the important role of the state in orchestrating and defining the policy 
issues at stake in the five Nordic countries under study. Moreover, as ana- 
lyzed in the Norwegian chapter by Hórmann and Sivesind (Chap. 7), 
references published by the government are important for linking to and 
legitimizing policies from previous reforms and arguments, retrospec- 
tively. In Iceland and in Denmark, there are also highly politicized pro- 
cesses, where either the minister has a clear opinion, as in Iceland (see 
Magningttir & Johansson, Chap. 6), or where political parties are 
involved in negotiations before a paper is written (see Juul Reder & 
Ydesen, Chap. 4). 

The number of references produced by research institutes that have 
had a traditional role of being knowledge suppliers to governments dif- 
fers among the Nordic countries. Internationally, the label research insti- 
tute encompasses many types of organizations that vary in the degree of 
how public they are (Late, 2019). They can according to Gulbrandsen 
(2011) be described as boundary organizations, as they often operate as 
agencies crossing the boundary of science and non-science (Late, 2019, 
p. 52). Not surprisingly, the use of references produced by such research 
institutes is central to both Norway's and Denmark's policymaking pro- 
cesses. In Norway, research institutes were initially established to per- 
form R&D areas of interest for sector authorities. As Baek et al. (Chap. 
9) and Steiner-Khamsi et al. (Chap. 10) demonstrate, the institute sector 
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is a highly important knowledge provider that conducts educational 
policy research and research-based evaluations in Norway. Likewise, as 
Juul Reder and Ydesens analysis in Chap. 4 depicts, the Danish 
Evaluation Institute (EVA) and the Danish National Research Centre 
for Social Research (SFI) represent two large agencies that produce pol- 
icy-relevant, specialized knowledge under the auspices of two public 
ministries. 

There is a striking difference between Norway and Denmark on the 
one side and Sweden and Finland on the other side. While the research 
institutes dominate in the two first, public agencies are more powerful 
as knowledge providers in the two last. In Sweden, the public agency 
Skoleverket serves the role of being the most important knowledge pro- 
vider. This agency generates official knowledge and research statistics 
about the school system and childcare in the country. The agency pro- 
duces policy-relevant data as well as research reports that are frequently 
referenced in the white papers in the Swedish data. A similar reference 
pattern seems to be the case in Finland. Volmari et al. (Chap. 5) under- 
score the strong expert position and power of the Finnish National 
Agency of Education.' This organization collects its own data and pro- 
vides analyses and evaluations. This governmental institution is in itself 
an expert body similar to the Swedish agency albeit shorter history as an 
independent agency. Nonetheless, Volmari et al. explain that two uni- 
versities in Finland have two publicly funded research centers that play 
a vital role in producing sector-based knowledge, not the least, OECD- 
funded studies, with a special responsibility for PISA. One of these 
institutes has been part of the EIPPEE network. We find similar centers 
in all Nordic countries that link the universities to globalizing policy 
spaces with a certain impact on policymaking processes in the Nordic 
countries. 

As mentioned, the scarcity of peer-reviewed academic references, 
including academic books and journals, is a striking feature of the refer- 
ence pattern in our documentation. In Chap. 9, Baek et al. demonstrate 
that 3096 of the references in the Nordic sample are journal articles and 
books. In Chap. 10, Steiner-Khamsi et al. present an analysis of the types 
of references that receive focus in the white papers and green papers in 
the Swedish and the Norwegian cases, using a detailed classification 
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system that distinguishes between both national and international aca- 
demic research references. They report that, while national and interna- 
tional academic research comprises about 4096 and 25% of the references 
in the Norwegian green papers and white papers, respectively, in the 
Swedish context, such research constitutes only 2096 of the references in 
green papers and none in white papers. To understand these variations, 
more in-depth analysis is needed. However, the numbers alone may point 
to the status of academic research in Sweden, where only 796 of the total 
number of references represented national academic research. 
Furthermore, as Wahlstróm and Nordin conclude in Chap. 8, we need to 
consider how intermediary organizations mediate academic research, 
summarizing and translating results and interpretations in a simplified 
and accessible way. 

Our analysis thus far does not provide much empirical knowledge for 
the five countries on the use of references from organizations that present 
themselves as think tanks. Christensen and Holst (2020) refer to think 
tanks as organizations that aim at influencing political debates and deci- 
sions by referencing knowledge and information. The think tanks can 
be considered as Rich (2004, p. 150) defines the term: “independent, 
non-interest-based, non-profit organizations that produce and princi- 
pally rely on expertise and ideas to obtain support and to influence the 
policy making process." Medvetz (2012, p. 213) extends this definition 
by defining think tanks as a “hybrid institutional area situated at the 
nexus of the political academic, economic and media fields.” 
Notwithstanding, in the Nordic countries, both as organizations and as 
discursive spaces, think tanks are relatively seen, new inventions, since 
the organizational figuration of actors as described in the literature on 
think tanks differs in comparison with the more established research 
institutes that provide sector research. 

One key disparity is that research institutes in the Nordic countries 
receive funding from the Research Council of Norway and conduct com- 
missioned research that are regulated by the contracts with public and 
private partners. While research institutes must follow formal contracts 
that to some extent prohibit them from taking an active part in political 
negotiations, think tanks pursue ideological agendas as discursive tools. 
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Therefore, research institutes do not act and react to political issues in the 
ways that think tanks can do. Therefore, think tanks claim to generate 
complementary knowledge in comparison with research carried out by 
universities and research institutes, both because of their institutional 
affiliation and due to their ways of deploying knowledge and expertise. 
Findings based on the POLNET study indicate that our documents cite 
think tanks only to a modest degree, if at all, in the reference lists. Yet, 
they deserve a closer look in future research, as they may become influen- 
tial through advocacy coalitions (Sabatier & Weible, 2007; Steiner- 
Khamsi, 2021) and policy assemblages where policy is produced through 
the ancillary of political and environmental contexts (Legrand, 2021). 

Taken together, the research and development systems (including 
innovation) and the policy advisory systems within the five Nordic coun- 
tries studied in this book have certain similarities, but the sites at which 
policy knowledge, such as research and evaluations, are produced do 
reflect significant variations. A lesson learned from reading the chapters 
of this book is that there are no clear boundaries between the sites for 
knowledge production and the sites for knowledge usage. This again leads 
to unclear separation of roles of policy makers on the one hand and pol- 
icy advisory commissions, including researchers, on the other hand. 


International Organizations: The Supremacy 
of the OECD 


As part of the rise of the evaluative state (Maroy, 2009; Neave, 1988, 
2009), research-based evaluations and surveys provide evidence and 
knowledge that are useful for legitimizing school reforms. Moreover, as 
Martens et al. (2016, p. 518) declare about the effects of this evolvement: 
“There is little doubt that international assessments established a ‘new 
center of gravity’ in the field of education and that they re-shaped educa- 
tion policy-making and practices in many countries.” For this and other 
reasons; universities, university colleges, and the various forms of research 
institutes act as competing or collaborating tenderers/bidders in seeking 
to manage various types of government-funded evaluations in the Nordic 
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countries. This is obviously the case in the Norwegian context, where the 
Ministry, following major educational reforms in 1990, decided to intro- 
duce research-based evaluation programs at the turn of the 2000s (see 
also Zapp et al., 2018). 

Besides the formal research and development systems that we consider 
as domestic knowledge providers, this book unravels the importance of 
international organizations as core knowledge producers in policymaking 
processes. As Steiner-Khamsi (2013) suggests, international knowledge 
and comparative studies in particular can influence policymaking pro- 
cesses (a) as evidence that informs policy planning within particular con- 
texts, (b) as normative guidelines for how to change educational processes 
concerning global problems, or (c) as projecting best practices that are 
evaluated against a set of international performance standards. A key 
point in Steiner-Khamsi's work is that the production of knowledge, 
especially the design of comparative research projects, optimizes evidence 
for the research impact on features that are not necessarily structured in 
the same ways across the selected sample of cases. Against this back- 
ground, we expected extensive references to comparative studies for legit- 
imizing reforms of certain features, independent of regional commonalities 
that are traditionally associated with the Nordic model. 

Based on our sample of documents, we find that OECD is the most 
significant knowledge producer of the international publications refer- 
enced in all the Nordic cases. In the comparative Chap. 11, Ydesen et al. 
conclude that the five country-specific cases reveal multiple layers in 
OECD-related references. Based on their analysis, the authors emphasize 
that the OECD policy instrument carries more weight than that found in 
a quantitative analysis of references only. By forming a powerful epis- 
temic space, the OECD is more powerful than all other international 
organizations. Nordic countries have a long-lasting tradition of being 
advised and supported by the OECD on how to govern their educational 
system (Ydesen, 2019). By the turn of the century, the attention of the 
OECD became extended by the PISA study, in which all five countries 
participated from the year 2000 (Sivesind, 2019). 

Verger et al. (2019) recently compiled a literature review on how the 
OECD influences education policy in diverse countries by governance 
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mechanisms and demonstrated how the organization has affected mem- 
ber countries differently by their way of collecting data, evaluating qual- 
ity, and generating ideas for how to improve education systems. 'Ihe 
POLNET study shows variations in the usage of OECD references. As 
illustrated by the Swedish case of Nordin et al. (Chap. 8), although the 
number of references to the OECD is modest compared with the num- 
ber of references to domestic and government sources, the Ministry 
turned explicitly to the OECD for analytical help to tackle its national 
school crisis in 2014. Likewise, in the Finnish case, as shown in Chap. 5, 
while the bibliometric analysis reveals evidence used in the 2014 curricu- 
lum reform was predominantly domestic and self-referential, a content 
analysis discloses the OECD and, especially, the PISA results were clearly 
visible in numbers in the policy documents. The same was demonstrated 
for the Icelandic case in Chap. 6; OECD evidence was used particularly 
to legitimize policy recommendations and design together with knowl- 
edge sources from Canada and a consultancy company, such as McKinsey. 

As Steiner-Khamsi (2013, p. 27) argued, not all forms of comparison 
will necessarily lead to policy borrowing and lending across countries. 
Policy transfer depends on various conditions, not least the methodologi- 
cal design used to construct the tests and surveys within OECD studies. 
Within the Nordic region of Europe, PISA studies, together with similar 
large-scale assessments, create what Waldow (2019) interestingly concep- 
tualizes as projections of best practices. This concept implies that refer- 
ences to best practices are not outcomes of particular conditions that 
regulate education policy, but rather socially constructed narratives that 
policy actors make to reduce the complexity they experience (pp. 4—5). 
Nonetheless, as noted by Steiner-Khamsi in Chap. 2, due to differences 
between the successful performance scores of Finnish students in PISA, 
the Finnish education has been glorified as one of the most successful 
systems in the world, and their results are outstanding also in comparison 
with their Nordic neighbors. Thus we ask: Given the noticeable varia- 
tions in performance scores between Finland and other Nordic countries, 
how are policy makers and experts referencing Finland within the pol- 


icy papers? 


420 B. Karseth and K. Sivesind 


Our material does not indicate that national publications about the 
Finnish education system and Finnish educational practices as such 
attract extraordinary attention by being highly referenced in our data- 
set. Interestingly, in the comparative chapter by Volmari et al. (Chap. 
12), documents from three Nordic countries refer to two national cur- 
ricula from Finland, whereas other Nordic curricula are not co-cited. In 
general, white and green papers from Iceland, Finland, and Norway 
reference national publications from Sweden and Denmark more often 
than Sweden and Denmark refer to publications from other Nordic 
countries. Overall, regional collaboration, as presented by the inter- 
viewees, does not seem to lead to policy convergence or diffusion albeit 
Finland is highlighted as an outstanding system. Rather, national 
authorities decide in their own contexts what to learn from Finland and 
others, following their own institutional paths for deciding what to do 
within their own contexts. This observation leads us to conclude that 
option (a) within the typology of Steiner-Khamsi seems to be the most 
typical feature of policy borrowing and lending among the Nordic 
countries, while we are also open to the possibility that OECD studies 
and transnational knowledge about performance scores on PISA influ- 
ence national education reforms more silently (Waldow, 2009). 

Possible reasons for the self-referentiality of school reform policy are 
the historical and organizational conditions that policy makers refer to 
and which differ between countries. Conditions are important levers in 
reforming schools, which implies the steady need to search for context- 
dependent knowledge that in the next step shapes the narratives of best 
practices. Moreover, such disparities may not constitute research objects 
or topics favored within international assessments because context- 
sensitive knowledge is neither necessarily capable of being measured by 
research items used for comparative studies, nor easily transferred between 
the contexts that we have studied. Therefore, in general, only those 
dimensions that are constructed as generic, that be, in terms of values and 
standards, are possible targets for international knowledge transfer. 
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The Knowledge Work of the Government 
and Its Agencies 


As the chapters in this book uncover, the state plays a vital role in policy- 
making processes in all the Nordic countries, and besides the ministries, 
semi-independent central agencies are core bodies that have a say in these 
processes (Greve et al., 2020). Although the level of independence of 
these agencies varies, their tasks are to obtain, translate, and implement 
reform ideas (Rovik et al., 2014). As already mentioned, public agencies 
are not per se an think tanks organization with political ambitions. Yet, 
they can organize international cooperation and activities, orchestrated 
by powerful policy actors, such as the OECD and the European Union 
(EU). Thereby, they function as agenda setters that mediate knowledge 
"from the outside" in policymaking processes that are regulated by the 
state. By coordinating and inviting others to join their research and devel- 
opment fora and to participate in various forms of knowledge work 
together with for example professional associations (Nerland and Karseth, 
2015), the agencies advise ministries on how to deploy both a repertoire 
of reform ideas and standards of which national policy makers and politi- 
cians should be aware. 

However, central agencies at the national level can also be viewed as 
gatekeepers in orchestrating ideas of how to translate, whom to involve, 
and how to connect topics and realms in policymaking processes. 
Although ministries mandate and oversee activities within boards and 
agencies, they can also be considered a hub that regulates its own knowl- 
edge flow between policy makers, international organizations, various 
stakeholders as well as researchers. This evolving layer of knowledge can 
be observed in how knowledge sources produced by the agencies are ref- 
erenced in governmental papers, and is therefore an interesting question 
what these agencies prioritize to produce as well as reference in their 
reports. 

The POLNET study has not identified reference patterns in reports 
produced by the state agencies. Yet, as some of the country chapters have 
demonstrated, national agencies seem to play an important role for pro- 
viding evaluations and assessments in their education systems, that 
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resonate policy makers and experts’ need for evidence. Simultaneously, it 
is important to recognize that the ministries themselves are both receivers 
and translators of this evidence (see Chap. 7). Anyhow, the governance 
arrangements of national agencies seem to be in a mode of constant 
change through processes of mergers between entities, organizational 
rearrangements, and new establishments (see Arlestig & Johansson, 
2020). Therefore, it is of interest to study how agencies and ministries 
interact in their search for policy solutions that potentially results in edu- 
cation reform. We may argue that ongoing transitions reflect the search 
for a workable and legitimate balance of responsibilities and tasks between 
politicians, bureaucrats, and experts and not at least, between ministries, 
boards/agencies and coalitions with stakeholders. There is a need to study 
the intersection between these bodies in future research. 

Nonetheless, by investigating reference patterns in our database, we 
observed that the importance of the state and the ethos of a robust public 
administration system still seem a characteristic of the Nordic mindset 
(see Volmari et al., Chap. 12). While processes for the deregulation, 
privatization, and marketization of Nordic education are ongoing, 
researchers like Dovemark et al. (2018) concluded that the changes are 
not as dramatic as those occurring in other countries. Nordic countries 
can be described, according to Maroy et al. (2017), as closer to a Neo- 
statist variant of managerialism than a neoliberal variant. Alternatively, as 
observed by Greve et al. (2020), a mixture of multiple reform packages is 
in use in Nordic countries that reflects institutionalized forms of coordi- 
nating the public sector, although managerial tools are at the forefront 
(p. 706). This leads in the next step to the conception of a “welfare mix” 
(Sivesind & Saglie, 2017) that characterizes the Nordic education sys- 
tems. Therefore, unraveling the complexity of governance structures and 
policy borrowing and lending across nations is imperative. Taken together, 
to understand the practice of evidence-based policy making, we need to 
capture how political institutions work: their procedures, routines, regu- 
lations, and relations. Moreover, further research on the central educa- 
tional agencies is needed to capture how science and politics are 
structurally coupled (see Steiner-Khamsi et al., Chap. 10; Steiner- 
Khamsi, 2021). 
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Evidence and Expertise in Transitions Between 
State Government and Network Governance 


The core aim of this book has been to explore distinctions in reference 
patterns between reform policies in the five Nordic countries. We have 
tried to unravel characteristics of what Eyal (2019, p. 33) labels *distrib- 
uted cognitions of expertise," that is, how expertise outside individuals is 
visualized through bibliometric patterns and reference networks. 
Certainly, it is not easy to uncover why some knowledge sources are 
selected by actors at the cost of others. Often, policy making is based on 
tacit, practical knowledge and an outcome of assemblages beyond con- 
scious decision making (Savage, 2020). Evidence-based policies can draw 
on knowledge from both the outside and inside of public policies. 
Therefore, the location of knowledge use is equally significant as the kind 
of knowledge produced and used. For that reason, excellent reasons exist 
for clarifying various reference patterns and for developing what Eyal 
(2019, p. 33) labeled explicit, abstract knowledge that expands and 
advances the sociology of expertise. This knowledge may well be medi- 
ated through books and articles that individual experts and others can 
read to enlighten conversations on public policies. Moreover, researching 
the sociology of expertise can provide knowledge that makes sense in 
contexts where people and bodies develop reforms on behalf of the state. 
In that case, various types of conditions can stimulate collaborative pro- 
cesses that help policy makers and experts to make recommendations on 
valid knowledge. 

Although not at the forefront of our analyses, the national and the 
comparative chapters have pointed to strategies that stakeholders deploy 
in policymaking processes (Chaps. 4 and 10). While public hearings have 
been important devices for recognizing various opinions and voices in 
public administrations that have been highly departmentalized (Sivesind 
& Skedsmo, 2020), today, many more channels are in use for collecting 
information and influencing policy. Organizations of stakeholder confer- 
ences, blogs, social media, the establishment of different types of refer- 
ence groups, think tanks, and public-private partnerships can evolve in 
various directions, mobilizing decision processes that call for new 
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constellations of innovative reference patterns that we could not unravel 
in our analysis of white and green papers and which need further exami- 
nation. The rearrangement of public policy implies that the modern state 
is "beset by a burgeoning array of domestic-global political, social and 
economic influence" (Legrand, 2021, p. 37). Ministries and state agen- 
cies are core bodies in this landscape of expertise, orchestrating meetings 
with national and international experts to communicate about kinds of 
evidence and expertise that are relevant for revising or renewing educa- 
tion policy. 

Simultaneously, new technology and co-governing strategies can in 
many ways give policy makers new opportunities to seek policy-relevant 
information. Crowd-sourcing procedures are nowadays used to collect 
information in reform processes from a range of actors; however, the 
respondents are not necessarily dedicated experts, specialized scientists, 
representatives of a particular knowledge field, or powerful stakeholders. 
Who is invited to participate is technically seen, contingent. Due to the 
ubiquitous access to digitalized information in society and new constella- 
tions of collaborators, there are good reasons to think of policy making as 
becoming pluralized. This pluralization of policy may transcend institu- 
tional boundaries that have guaranteed corporate decision making into a 
discursive policy space that allocates the attention of policy makers to 
new agenda-setters in education policy. 

Nordic countries are known for their corporative traditions where the 
state and the government have granted access to certain types of interest 
groups and organizations in arrangements such as public advisory bodies 
(Aberg et al., 2019). Such arrangements have induced both the stability 
and legitimacy of policies within civil society that in our case include 
powerful organizations, such as teacher unions. However, there are good 
reasons for seeing the policy system as well as traditions for developing 
professional expertise potentially transformed by evidence-based policy 
and thereby challenged. Therefore, to create legitimacy for changing poli- 
cies in the field of education, the complexity of both conditions and 
expectations urges academic enquiry. 

As Eyal (2019, p. 36) acknowledges, the application of expertise 
depends on “being connected with a network of expertise composed of 
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other actors, devices and instruments, concepts, and institutional and 
practical arrangements, distributed in multiple loci, yet assembled into a 
coherent, collective agency." This agency must deal with self-referential 
problems about the reflective use of policy knowledge that, in our case, 
refers to state-authorized school reforms. However, in our time, it must 
also look outside its own boundary to seek solutions to global problems 
in collaboration with others. Therefore, the international orchestration of 
policy spaces and the interdependent matrix of processes between various 
knowledge providers, deserve researchers' attention in future research. 


Note 


1. This agency replaced the Finnish national board of education in 2017 as 
it merged with CEMO (the Centre for International Mobility). This shift 
implies a transition from serving a role as a Directorate for Education to 
an agency within the national administration of education and training 
that has a two-tier structure similar to the Swedish case. 
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